Skip to content

Data ​

A page in this section says where each kind of data the comparison reads comes from, how Tidemark fetches and keeps it, and what each column it keeps holds.

Each of the three lengths The Unsurveyed Gauge adds up as h+d=zh + d = z comes from data of its own. The SWOT samples of the Level 2 files give the height of the sea surface above the WGS 84 ellipsoid, the satellite's reading of h before it is brought onto NAVD 88, whereas the Level 3 samples give its anomaly beside the terms that add back to it. Each gauge's series gives the water surface elevation above NAVD 88 its network publishes or, at a USGS gauge, works out from its gage height, or at Bar Harbor above mean lower low water, from which the distance d follows at a gauge with a surveyed elevation. The station catalog keeps a column for z, the elevation a survey establishes for a sensor.

The table below names each kind of data the project keeps, the side of the comparison it serves, and how a clone comes to hold it, written from the declaration in tidemark.docs.holdings by mise run docs:render.

From the pierMaine’s tide gauges

From orbitThe SWOT satellite

Each gauge's seriesFetched

The Gauge Series

Every water level a gauge has published, with its quality flags. Fetched into data/gauges/ from the NERACOOS ERDDAP, NOAA CO-OPS, and the USGS Water Data API.

bash
mise run data:fetch:gauges

Fresh cloneStep 3 of 3

Fills the SWOT passes under data/swot/.

Every value as a table
DataSideHow a Clone Holds It
The station catalogGaugeShips with the clone in src/tidemark/gauges/, rebuilt by mise run data:catalog, and mise run data:catalog -- --check compares it with a rebuild from the answers the NERACOOS ERDDAP⁠1⁠, NOAA CO-OPS⁠2⁠, and the USGS Water Data API⁠3⁠ gave the last one
Each gauge's seriesGaugeFetched into data/gauges/ from the NERACOOS ERDDAP⁠1⁠, NOAA CO-OPS⁠2⁠, and the USGS Water Data API⁠3⁠ by mise run data:fetch:gauges
The SWOT passesSatelliteFetched into data/swot/ from PO.DAAC⁠4⁠ by mise run data:fetch:swot, with a free NASA Earthdata login, which mise run data:login:earthdata saves, and from AVISO⁠5⁠ by mise run data:fetch:swot l3, with a free AVISO+ login, which mise run data:login:aviso saves
The pass scheduleSatelliteShips with the clone in src/tidemark/swot/ as tables computed from the SWOT passes a fetch keeps⁠4⁠, rebuilt by mise run data:schedule, and mise run data:schedule -- --check compares them with a rebuild

The Shared Key ​

Three columns join one table to another, each named the same way in every table carrying it, so a gauge's rows in one table meet its rows in the next, and each table on the pages below marks them as a shared key rather than describing them again.

  • station

    Which gauge a row is about, named as the station catalog files it

  • cycle_number

    Which repeat of SWOT's ground track a row comes from

  • pass_number

    Which pass a row comes from, a number it keeps every cycle

A mark shows a table holding that key, which joins it to every other table holding it. Reach a key above to light those tables, or press a table to go to its section.

Every value as a table
ColumnHolds
stationWhich gauge a row is about, named as the station catalog files it
cycle_numberWhich repeat of SWOT's ground track a row comes from
pass_numberWhich pass a row comes from, a number it keeps every cycle
Commit47945eaDatastations.jsonUnreleased1.2.0

The Data Directory ​

Each source's files, and the record of each run, sit in their own directory under data/, which git ignores, so nothing a fetch writes, the stakeholder shares, or a run records enters the repository. mise run data:clean clears what a fetch left behind there and keeps data/provided/ and data/runs/.

A file the stakeholder shares reaches a clone by hand rather than through a fetch, requested from the authors and placed under data/provided/, which mkdir -p data/provided creates in a fresh clone. A gauge's series takes the form The Stakeholder's Series sets out.

The Weekly Release ​

Each weekly release carries a Zstandard tar of what each fetch keeps, one for the service answers the station catalog was last rebuilt from, one for the OpenStreetMap coastline, one for the NOAA stations, one for the ERDDAP gauges, one for the USGS gauges, one for the GEOID18 grid, one for NGL's table, and one for each SWOT file, Level 2 and Level 3, each opening on a CREDITS.md naming its source and quoting its terms, beside a NOTICE.md holding the notice of every source whose terms are established and of every license. On release day, mise run gha:roll fetches every source, packs the tars through mise run data:pack, puts them on the week's draft release, and publishes it once main still heads the commit they were packed from. Before it packs, mise run data:catalog -- --check rebuilds the station catalog from the answers parts.json records, and before anything goes on the draft, mise run data:schedule -- --check and mise run data:excerpt -- --check rebuild the pass schedule and the example pass from exactly the parts parts.json records against the store just packed, so the release carries everything the shipped tables were built from, and the draft takes the site the 🌊 Pages workflow built from that commit as site.tar, which The Sandbox replays a release against. A tar past the 2 GiB GitHub allows a release file is written in numbered pieces.

A pack writes a tar again only where what it was packed from has changed or the tar is not whole. Beside each tar it records the fingerprint of each directory the tar holds (the text of its manifest.json and the SHA-256 digest of every other file), the tar's credits, a digest of the packing code, and the Zstandard version, keeping a whole tar whose record reads the same. Since the same files pack into the same bytes, mise run gha:attach then reads the draft again, keeps each file it carries whole under the packed file's digest, deletes the tar's other files, a piece past a shrinking count among them, and uploads only what the draft lacks, so the roll publishes once the draft carries exactly the pack's files beside the site.

A fetch reads through gh the newest release carrying its tar, passing over any release later than the version of the package it runs from, and seeds what data/ lacks before it downloads anything from osmdata, NOAA, PO.DAAC, AVISO, PROJ, or NGL, or asks the USGS for any reading, as The First Fetch and Seeding From a Release set out, and a SWOT fetch reads its source's list of passes ahead of that seed, since that list names the passes the seed fills. Under --release, a fetch of the SWOT passes or the NOAA stations instead seeds from the one release a tag or a day names, listing and fetching nothing, and the ERDDAP and USGS gauges stay out of that seed. A SWOT tar holds each file's passes and its manifest.json, whereas no tar holds the samples table, which every fetch fills again from the passes, and the Level 3 tar holds the parts the fetch cuts from AVISO's product with what the pipeline adds to them rather than its files as AVISO serves them, as The AVISO License sets out. Release 1.2.0 and each one after it carries these modified parts beside ssha_unedited, whereas releases 1.0.0 and 1.1.0 keep their original Level 3 tars, AVISO's product as the fetch cut it, which a replay of either tag reads, and mise run data:pack -- --release 1.1.0 writes the modified tar of either locally. No tar holds data/provided/ or data/runs/. A gh holding no login that reads the private repository, which gh auth login saves, leaves the fetch naming why it seeded nothing and asking each source for everything, whereas a fetch under --release fails once, naming the release, and seeds nothing.

A source can revise what an earlier release carries, as NOAA does when it verifies a month, the USGS when it approves a provisional reading, and AVISO when it reprocesses a Level 3 pass, and each published release keeps what it shipped with. The fetch ahead of the next release replaces whatever its own checks show changed at a source, and records each replacement in the revisions.json beside the directory's manifest.json, which the tar carries, naming the range or the pass, the instant of the fetch, and what showed the revision:

SourceReplaced OnceRecorded As
NOAA's CO-OPSNOAA has verified a month since, or a month's verified monthly mean has changedverified or mean
The NERACOOS ERDDAPA reading the ERDDAP revised or received late changed the tally of a run of days before the newest reading's daytally
The USGS Water Data APIUSGS modified a reading on a day before the newest reading's day, as it does when it approves a provisional onemodified
PO.DAAC's Level 2 productThe archive lists a pass's file under a newer CRID or product countercrid or counter
AVISO's Level 3 productThe catalog lists a pass under reproc whose part reads forwardreproc

A reader comparing two releases therefore tells a revision in the data from a change in method through the replacements fetched after the earlier release was published, and a fetch seeding from a release takes the release's replacements beside its files. A preliminary NOAA month read again on every rerun is replaced with no entry, since only a verification or a changed mean counts, and a change of request, such as another reach, empties a directory's record with its files. A revision these checks miss, such as a NOAA correction to a reading's flags that leaves its month's verified mean as it was, or a correction a source announces, reaches the next release once mise run gha:attach -- --refresh asks every source for every file again ahead of it, or once mise run gha:roll -- --refresh does on release day, which carries the revision into the data with no entry, since only what the checks show enters the record.

Credits ​

Each source's notice below names who produced its data, the acknowledgment its terms ask of work using those data, and where those terms are published, quoting each passage a copy shared onward keeps. licenses.toml at the repository's root writes this list, each tar's CREDITS.md, and the release's NOTICE.md, and the site's CREDITS.md sets out each license.

  • SWOT Level 3 at AVISO⁠5⁠. The product is produced by AVISO/DUACS and licensed by CNES under the AVISO License, and nothing here implies that CNES or AVISO endorses Tidemark. "The SWOT L3_LR_SSH product, derived from the L2 SWOT KaRIn low rate ocean data products (NASA/JPL and CNES), is produced and made freely available by AVISO and DUACS teams as part of the DESMOS Science Team project." CNES licenses the product under the AVISO License, https://www.aviso.altimetry.fr/fileadmin/documents/data/License_Aviso.pdf. Tidemark modified the product, so each part is a Value-Added Product rather than a copy. tidemark fetch swot l3 cuts each part through AVISO's DAP2 service to the lines and the pixels of one pass within reach of a gauge, the variables Tidemark keeps, and four of the file's global attributes. tidemark pack then adds four variables to each part. correction sums mss, ocean_tide, dac, and internal_tide. height adds ssha_unfiltered to correction. gauge_distance holds each sample's distance to the nearest gauge. coastline_distance holds each sample's distance to the OpenStreetMap coastline. It also adds global attributes naming these modifications, the producer, the license, and the digest of that coastline. The terms at https://www.aviso.altimetry.fr/fileadmin/documents/data/License_Aviso.pdf read, in part, "You must not Share the Licensed Material, in its original, unmodified form, through any operational platform intended for onward mass distribution — including but not limited to a web portal, public API, or cloud data infrastructure — without prior written authorization from CNES."
  • Nevada Geodetic Laboratory⁠6⁠. The table is produced by the Nevada Geodetic Laboratory (NGL) at the University of Nevada, Reno, from the GNSS stations whose data it processes. The terms at https://geodesy.unr.edu/ read, in part, "Henceforth please use the following for citation of our data products: Blewitt, G., W. C. Hammond, and C. Kreemer (2018), Harnessing the GPS data explosion for interdisciplinary science, Eos, 99, https://doi.org/10.1029/2018EO104623." "Please also note that many stations have an original data citation that should also be referenced."
  • U.S. Census Bureau's cartographic boundary files⁠7⁠. The U.S. Census Bureau publishes the states' boundaries, simplified for display at 1:500,000 and clipped to the shoreline. The terms are at https://www.usa.gov/government-copyright.
  • NOAA CO-OPS⁠2⁠. The data are NOAA's, produced by the National Ocean Service's Center for Operational Oceanographic Products and Services (CO-OPS). The terms at https://tidesandcurrents.noaa.gov/disclaimers.html read, in part, "NOS requests that attribution be given whenever NOS material is reproduced and re-disseminated." "These raw data have not been subjected to the National Ocean Service's quality control or quality assurance procedures and do not meet the criteria and standards of official National Ocean Service data. They are released for limited public use as preliminary data to be used only with appropriate caution."
  • NERACOOS ERDDAP⁠1⁠. Each dataset names the Gulf of Maine Research Institute (GMRI) or Brown University as its institution and declares the license quoted below on its own page, as the Bath gauge's dataset does, and NERACOOS serves them through its ERDDAP. These research findings were informed in part by data provided by the Northeastern Regional Association of Coastal Ocean Observing Systems (NERACOOS) and accessed through NERACOOS's ERDDAP. NERACOOS is a component of the U.S. Integrated Ocean Observing System. The terms at https://data.neracoos.org/erddap/info/Hohonu_tide_Bath_ME/index.html read, in part, "The data may be used and redistributed for free but is not intended for legal use, since it may contain inaccuracies. Neither the data Contributor, ERD, NOAA, nor the United States Government, nor any of their employees or contractors, makes any warranty, express or implied, including warranties of merchantability and fitness for a particular purpose, or assumes any legal liability for the accuracy, completeness, or usefulness, of this information."
  • Natural Earth⁠8⁠. Natural Earth publishes the land, the minor islands, and the lakes the site's cards draw beyond the Gulf of Maine. Made with Natural Earth. Free vector and raster map data @ naturalearthdata.com. In the public domain. The terms at https://www.naturalearthdata.com/about/terms-of-use/ read, in part, "All versions of Natural Earth raster + vector map data found on this website are in the public domain."
  • NOAA's ETOPO1⁠9⁠. NOAA's National Geophysical Data Center produced the model, which its National Centers for Environmental Information publish. Amante, C. and B.W. Eakins, 2009. ETOPO1 1 Arc-Minute Global Relief Model: Procedures, Data Sources and Analysis. NOAA Technical Memorandum NESDIS NGDC-24. National Geophysical Data Center, NOAA. doi:10.7289/V5C8276M. The terms are at https://www.usa.gov/government-copyright.
  • GEOID18 at PROJ's CDN⁠10⁠. GEOID18 is produced by NOAA's National Geodetic Survey (NGS), which publishes it at https://geodesy.noaa.gov/GEOID/GEOID18/, and the PROJ project serves it converted to GeoTIFF. The terms at https://cdn.proj.org/us_noaa_README.txt read, in part, "License: Public Domain"
  • NGS's HTDP⁠11⁠. NOAA's National Geodetic Survey publishes the tool and the velocity model it moves a position through, which the heights page compares against without shipping any of its output. The terms are at https://www.usa.gov/government-copyright.
  • NGS Data Explorer⁠12⁠. NOAA's National Geodetic Survey serves each survey mark's datasheet through its Data Explorer API. The terms are at https://www.usa.gov/government-copyright.
  • VDatum⁠13⁠. NOAA's National Ocean Service publishes VDatum, whose web service gives each conversion beside its uncertainty. The conversions between vertical datums obtained through the use of original VDatum originate with NOAA/NOS. The terms are at https://www.usa.gov/government-copyright.
  • OpenStreetMap coastline⁠14⁠. The coastline is derived by osmdata.openstreetmap.de, through osmcoastline, from the ways OpenStreetMap's contributors tag natural=coastline. © OpenStreetMap contributors, available under the Open Database License (ODbL), https://opendatacommons.org/licenses/odbl/1-0/. The package and the site ship two tables derived from it, the cut of the coastline in each release and each sample's distance to the coast in the example pass, and each Level 3 part a release carries holds a third, its coastline_distance, so Tidemark offers them under the same license, as its share-alike terms set out, and a table a run derives from the coastline takes the same terms wherever it is shared, the samples table for its coastline_distance column and the coasts table whole, every value of which is computed from those distances. mise run data:fetch:coast makes the cut. The terms at https://www.openstreetmap.org/copyright read, in part, "OpenStreetMap is open data, licensed under the Open Data Commons Open Database License (ODbL) by the OpenStreetMap Foundation (OSMF)." "You are free to copy, distribute, transmit and adapt our data, as long as you credit OpenStreetMap and its contributors. If you alter or build upon our data, you may distribute the result only under the same license."
  • GMRT⁠15⁠. The Global Multi-Resolution Topography Synthesis is hosted at the Lamont-Doherty Earth Observatory of Columbia University. Depths and heights from the Global Multi-Resolution Topography Synthesis (GMRT), https://www.gmrt.org/, described by Ryan et al. in Geochemistry, Geophysics, Geosystems 10(3), Q03014, and licensed under CC BY 4.0, https://creativecommons.org/licenses/by/4.0/. Tidemark modified them, tracing closed steps of depth and height from the grid as lines. The terms at https://www.gmrt.org/about/terms_of_use.php read, in part, "Appropriate acknowledgment with a byline/credit/link must be given to the Global Multi-Resolution Topography Synthesis (GMRT)."
  • Statistics Canada's cartographic boundary file⁠16⁠. Statistics Canada publishes the provinces' boundaries as of the census's geographic reference date. Adapted from Statistics Canada, Provinces/Territories, Cartographic Boundary File, 2021 Census, January 1, 2021. This does not constitute an endorsement by Statistics Canada of this product. The terms at https://www.statcan.gc.ca/en/reference/licence read, in part, "Where any Information is contained within a Value-added Product, you shall include on such Value-added Product the following notice: Adapted from Statistics Canada, name of product, reference date. This does not constitute an endorsement by Statistics Canada of this product."
  • SWOT Level 2 at PO.DAAC⁠4⁠. SWOT's data are produced by NASA, CNES, the Canadian Space Agency, and the UK Space Agency, and distributed through NASA's Physical Oceanography Distributed Active Archive Center (PO.DAAC). The terms at https://www.earthdata.nasa.gov/earth-observation-data/data-use-policy read, in part, "Unless the content is marked with a use restriction or license, data provided from a NASA-led mission are licensed as Creative Commons Zero (CC0). While there are no restrictions on the use of these data, data users are very strongly urged to cite the data used in their work products."
  • USGS Water Data API⁠3⁠. The data are produced by the U.S. Geological Survey (USGS), whose provisional data statement at https://waterdata.usgs.gov/provisional-data-statement/ opens: "This information is preliminary or provisional and is subject to revision." Credit: U.S. Geological Survey. The terms at https://www.usgs.gov/information-policies-and-instructions/copyrights-and-credits read, in part, "USGS-authored or produced data and information are considered to be in the U.S. Public Domain." "When using information from USGS information products, publications, or websites, we ask that proper credit be given."

The coastline this page reads is a derivative database under the Open Database License, and tidemark fetch coast makes a free copy of the cut.

We read the distances as a derivative database in the license's terms, because each one is measured against the whole cut, which has far more than the 100 coastline features the OpenStreetMap Foundation's Substantial Guideline counts as insubstantial. Each is kept as data to extract rather than as a map or an image, which the Foundation's Produced Work Guideline sets apart as a produced work. The SWOT values beside them include no OpenStreetMap data, so under the Foundation's Collective Database Guideline the table is a collective database and the share-alike terms reach the distances alone.

The Pages ​

Inside the Station Catalog follows the table of every gauge back to the services behind it, and the first steps of The Analysis Graph read its anchors and its targets.

The Gauge Series then follows each of those gauges' water levels from the service that publishes them into one table.

The Satellite Passes follows each SWOT pass near a gauge from PO.DAAC and AVISO into a table of samples for each of the three products it keeps, and The Pass Schedule measures each gauge's place across the swath from its nadir tracks.

The Service Records closes the section on the record each service answers in, for a reader adding a source.

Each kind of data brings an error of its own into the comparison, and Studies sets out how other teams have measured and worked around each place one can enter.


  1. 1Gulf of Maine Research Institute & Brown University 2026. "NERACOOS ERDDAP." Northeastern Regional Association of Coastal Ocean Observing Systems. https://data.neracoos.org/erddap/.↩︎ ↩︎ ↩︎

  2. 2NOAA CO-OPS 2026. "CO-OPS Data Retrieval API and Metadata API." https://tidesandcurrents.noaa.gov/web_services_info.html.↩︎ ↩︎ ↩︎

  3. 3United States Geological Survey 1994. "USGS Water Data for the Nation." https://doi.org/10.5066/F7P55KJN.↩︎ ↩︎ ↩︎

  4. 4SWOT 2025. "SWOT Level 2 KaRIn Low Rate Sea Surface Height Data Product." Version D. NASA Physical Oceanography Distributed Active Archive Center. https://doi.org/10.5067/SWOT-SSH-D.↩︎ ↩︎ ↩︎

  5. 5AVISO/DUACS 2024. "SWOT Level-3 SSH Unsmoothed." Version 3. CNES. https://doi.org/10.24400/527896/A01-2024.003.↩︎ ↩︎

  6. 6Blewitt et al. 2018. "Harnessing the GPS Data Explosion for Interdisciplinary Science." Eos 99. https://doi.org/10.1029/2018EO104623↩︎

  7. 7U.S. Census Bureau 2024. "2024 Cartographic Boundary File (SHP), State and Equivalent Entities for United States, 1:500,000." https://www2.census.gov/geo/tiger/GENZ2024/shp/cb_2024_us_state_500k.zip.↩︎

  8. 8Natural Earth 2022. "Natural Earth 1:10m Physical Vectors: Land, Minor Islands, and Lakes." Version 5.1.1. https://www.naturalearthdata.com/downloads/10m-physical-vectors/.↩︎

  9. 9NOAA National Geophysical Data Center 2009. "ETOPO1 1 Arc-Minute Global Relief Model." NOAA National Centers for Environmental Information. https://doi.org/10.7289/V5C8276M.↩︎

  10. 10National Geodetic Survey 2019. "GEOID18." PROJ. https://cdn.proj.org/us_noaa_g2018u0.tif.↩︎

  11. 11National Geodetic Survey 2025. "HTDP – Horizontal Time-Dependent Positioning." Version 3.6.0. https://geodesy.noaa.gov/TOOLS/Htdp/Htdp.shtml.↩︎

  12. 12National Geodetic Survey 2026. "NGS Data Explorer API." https://geodesy.noaa.gov/web_services/data-explorer.shtml.↩︎

  13. 13NOAA National Ocean Service 2026. "VDatum." https://vdatum.noaa.gov/.↩︎

  14. 14OpenStreetMap contributors 2026. "Coastlines." osmdata.openstreetmap.de. https://osmdata.openstreetmap.de/data/coastlines.html.↩︎

  15. 15Ryan et al. 2009. "Global Multi-Resolution Topography Synthesis." https://doi.org/10.1029/2008GC002332.↩︎

  16. 16Statistics Canada 2021. "Provinces/Territories, Cartographic Boundary File, 2021 Census." https://www12.statcan.gc.ca/census-recensement/2021/geo/sip-pis/boundary-limites/files-fichiers/lpr_000b21a_e.zip.↩︎