← Astronomy 16 / 18

The data

Ten megabytes live in resources/astro: eight planetary series, a lunar theory, a nutation series, a table of mean elements, delta T, the masses of the solar system, seven tabulated small bodies and a catalogue of 1,099 fixed stars. Every number in every one of those files came from a publication, not from a keyboard.

There is not a single hand-typed coefficient in any of them, and that is a design rule, not a boast. Fifty thousand numbers went into building this engine, and one mistyped digit gives a position that is quietly wrong: a chart with Mars three degrees off looks exactly as plausible as one that is right, because nothing about a wheel of glyphs announces which planet is lying. So nothing here is typed. Twelve commands download the published sources and write the tables, and running one again reproduces its own file byte for byte. There is exactly one exception, and it says so in capitals at the top of its own file, told at the end of this chapter.

Twelve commands, and what each one writes

vendor/bin/astronomy vsop87
vendor/bin/astronomy elp2000
vendor/bin/astronomy delta-t
vendor/bin/astronomy nutation
vendor/bin/astronomy mean-elements
vendor/bin/astronomy masses
vendor/bin/astronomy magnitudes
vendor/bin/astronomy tables pluto
vendor/bin/astronomy moon-correction
vendor/bin/astronomy planet-correction
vendor/bin/astronomy stars
vendor/bin/astronomy satellite-list
Command Class Where the numbers come from What it writes
vsop87 Vsop87Series VSOP87D, Bretagnon and Francou, published at the CDS in Strasbourg vsop87/*.php, one file per planet
elp2000 Elp2000Series ELP 2000-82B, Chapront-Touzé and Chapront, also at the CDS elp2000/*.php, one file per coordinate plus its constants
delta-t DeltaTTable the Stephenson, Morrison and Hohenkerk 2016 spline before 1955, the USNO to 1973, the IERS after that deltat.php, with the leap seconds inside
nutation NutationSeries ERFA's nut00b.c and the five fa*03.c Delaunay argument polynomials, BSD licensed nutation.php
mean-elements MeanElementsTable ERFA's plan94.c, Simon and others, A&A 282 (1994) mean-elements.php
masses MassTable the ASCII header of DE440, published by the JPL masses.php
magnitudes MagnitudeFit fitted against JPL Horizons' own published magnitudes magnitudes.php
tables <body> PositionTables JPL Horizons, heliocentric vectors positions/<body>.php, one per tabulated body
moon-correction MoonCorrectionTable JPL Horizons, geocentric vectors of the Moon correction/moon.bin
planet-correction PlanetCorrectionTable JPL Horizons, heliocentric vectors of the eight planets correction/<body>.bin
stars StarCatalog SIMBAD for identity, Hipparcos-2 through VizieR for astrometry stars.php
satellite-list SatelliteList the JPL Horizons major bodies list the cases of the Satellite enum, in source

Two of those, tables and satellite-list, are named apart from their siblings on purpose. tables is not positions, which is what Downloader writes for an asteroid somebody asked for in downloads/: this one rebuilds one of the seven tables the engine itself reads to place Pluto, Chiron, Pholus and the four asteroids it knows by name, and positions/<body>.php is a different folder from downloads/. satellite-list rewrites this package's own source rather than fetching a satellite's positions, which is what astronomy satellites Io does; see Downloadable bodies for that half.

What it weighs

du -sh resources/astro/*

4.3M	resources/astro/correction
32K	resources/astro/deltat.php
260K	resources/astro/elp2000
20K	resources/astro/fictitious.php
8.0K	resources/astro/magnitudes.php
4.0K	resources/astro/masses.php
8.0K	resources/astro/mean-elements.php
8.0K	resources/astro/nutation.php
4.0M	resources/astro/positions
136K	resources/astro/star-names.php
524K	resources/astro/stars.php
828K	resources/astro/vsop87

Ten megabytes, and 4.3 of them are the correction towards Horizons and 4.0 the seven tabulated bodies, which between them are most of what a birth chart needs beyond the eight planets. That is the whole cost of not depending on anybody else's server the day this deploys: no migration, no row in a database, no cache to warm. Astronomy\DataFolder decides where they are read from, and by default that is this folder, deduced from where the class itself sits, so nothing has to be configured to use what ships. An application that keeps its data somewhere else calls DataFolder::useFolder() once at boot, before anything is read; calling it again after something has already been read throws, because two folders read from at different points in the same request would mean answering the same question two different ways without anybody choosing to.

The check is what a generator writes

A generator that downloads and parses correctly can still be wrong in a way nothing reports: a truncated response, a column read one place over, a threshold left at its default. None of those raise an exception. They write a file that loads fine and answers every call with a number, just the wrong one. So each of the twelve checks itself against the only thing that means anything for a generator, which is what it was told to produce:

  • Vsop87Series reads the announced term count out of each block's own header (367 TERMS) and throws if what it parsed does not match, rather than trusting that the file ended where it should have.
  • NutationSeries and MeanElementsTable parse ERFA's C source with a fixed count to find (77 luni-solar terms, six tables of mean elements with three coefficients each) and refuse to write anything if the count is short.
  • StarCatalog resolves every star it is asked for by a cone search around a published position, and if a single one does not resolve, nothing is written at all: a catalogue silently missing one star gives no error anywhere, that star just stops appearing in charts and nobody notices.
  • DeltaTTable cross-checks the years where the USNO and the IERS overlap, 1974 to 1984, against each other before writing either.

And underneath all of that is the same claim the README opens with: every one of the twelve was run against the network and its output compared byte for byte with the file already in the repository. Regenerating with no changes upstream reproduces the same bytes, which is what lets the check mean something; if it did not, a generator could be silently different every time it ran and nobody would know from the tests, which never touch the network at all.

A threshold that changed 621 terms into 213 without failing anything

Vsop87Series::THRESHOLD is the amplitude in radians below which a term is dropped, because a term smaller than that cannot move the answer to the precision that matters here:

/**
 * Amplitude below which a term is dropped, in radians (and in AU for the radius).
 *
 * It is 1e-8 because that is what the series that ship were built with, and a default
 * that does not reproduce them is a command that quietly makes the engine worse: the console
 * command this came from defaulted to 1e-7 and the committed tables were written by passing
 * 1e-8 by hand, so running it with no options rewrote Earth with 213 terms where it has 621,
 * discarded 2,212 instead of 1,804, and failed at nothing. A chart drawn from that is not
 * wrong in a way anybody sees; it is just less true.
 */
public const THRESHOLD = 1e-8;

Nothing in that run threw. The file it wrote had a well-formed header, a term count matching that header, and every downstream test passed, because the tests fix numbers to a tolerance loose enough that a truncated Earth still falls inside it. The only thing that caught it was running the command with no options and diffing the result against the file already committed. Elp2000Series carries the same constant for the same reason, and it is exactly the trap a threshold left at its library default always sets: the number that used to be right quietly becomes the number nobody chose.

It is also not worth pushing further than 1e-8. Measured on the Moon, going one order of magnitude finer changes the raw residual against a modern integrated ephemeris from 15.05 arcseconds to 15.07, which is to say it does not change it: what is left there is not truncation any more, it is the forty-year-old fit itself, and no amount of extra terms buys anything against that. Tripling the series would triple the cost of every Moon this engine computes, which eclipses and occultations ask for thousands of times over a search, to save at most a hundred kilobytes of table.

What gets stored is what is missing, not the position

VSOP87 and ELP are analytical series fitted decades ago, and against a modern numerically integrated ephemeris they drift: tenths of an arcsecond for the inner planets in the twentieth century, several arcseconds for Uranus and Neptune three centuries out, and the Moon sixteen arcseconds in 1600 because of how its tidal acceleration term was fitted. CorrectionTable, written by MoonCorrectionTable and PlanetCorrectionTable, does not replace the series with the correct position. It stores what the series is missing, not the series's replacement: storing the JPL's whole position for ten bodies over eight hundred years would cost tens of megabytes, while storing only the difference costs a fraction of that, because the difference is a number a thousand times smaller and therefore needs far fewer coefficients for the same absolute accuracy, which is the one that matters to a chart.

That is a Chebyshev fit in blocks, one per rectangular coordinate, stored as four-byte floats rather than eight, and the choice is measured rather than assumed: the correction of Uranus reaches 7e-4 AU, and the relative precision of a four-byte float is about 1e-7, which is 1e-10 AU of rounding error, four orders of magnitude below what is being corrected. Storing a whole position at that precision would be reckless; storing a correction of that size, it is not.

And it fails the way every generated file here has to fail: quietly wrong rather than loudly broken, so the format is built to make that impossible instead.

  • A file cut short does not error, it gives zero correction in the part that is missing. CorrectionTable's header carries the number of blocks, and the class checks the file's actual size against what that number implies before trusting a single coefficient in it.
  • The block choice is measured, not assumed, and the wrong choice looks fine. With a coarse sample, a block of forty-eight days at degree twenty-four fits forty-nine points against twenty-five coefficients, and the fit's error at its own points is 0.024 arcseconds while the real error, measured between those points, is 0.244: ten times worse, and the fit that produced it looks better than the one that did not. Sampling every six hours instead of once a day raises the same block to sixty-five points, admits degree fourteen, and the true error falls to 0.006 arcseconds. What is used is sixteen days, degree fourteen, a point every six hours, 3.14 megabytes for the whole 1600 to 2400 span.
  • Regenerating cannot check itself against what it just wrote. The residual MoonCorrectionTable measures against is Moon::spherical, the raw ELP series that knows nothing about a correction table, never Ephemeris, which already adds the correction in. Measuring against Ephemeris would mean the second run finds almost nothing left to correct and writes a table of zeros: the correction erasing itself, silently, the moment anybody regenerated it.
  • The table starts one block before the range it announces, because the engine looks slightly backwards for light time and takes a centred difference for velocity, both of which ask for the correction a little before the instant requested. Right at the first date of a table that falls outside its own span: measured, with a table starting exactly on the first of a year, that one day comes out at 0.134 arcseconds while the rest of the year sits at 0.04. No error anywhere, just one date quietly worse than its neighbours.

The one exception

resources/astro/fictitious.php carries the orbital elements of nineteen bodies that do not exist: eight the Hamburg school postulated in the 1920s and 1930s, four more with a following of their own, the four positions Le Verrier, Adams, Lowell and Pickering calculated before Neptune and Pluto were actually found, and three from discarded or pseudoscientific sources. Nobody has ever observed any of them, so there is no observation a command could fetch: what exists is a published table someone once wrote down, and reading it is the whole job.

The file's own header says why it is hand-written, in capitals, because it is the only place in the engine where that is true: "this file is HAND-WRITTEN, and it is the only exception in the whole engine." Nowhere else is there a single typed-in coefficient; VSOP87, ELP, nutation, delta T, the stars and the JPL tables are all downloaded from their source with a command and converted. Not here, and the reason proved itself while writing this file: the address the eight Uranian bodies' elements were once downloaded from no longer exists, and answers with a 404 showing a drawing of Pluto. A command that downloads a hundred and fifty numbers off a web page is a command that breaks the day the page moves, all to save typing them once in a lifetime.

What replaces a download here is two independent copies checked against each other: the file it was transcribed from is read from two separate mirrors that carry it in full, and the two agree character for character on every line this engine uses. That is not the standard the twelve generators above hold themselves to, and the file says so rather than pretending it is: it is the closest this one exception gets to the same guarantee.

Where to go next

Positions, orbits and phenomena are what these files feed. Downloadable bodies is the data that does not ship, fetched the same way but at request time and by whoever calls, never by the engine on its own. And Precision is what all of this adds up to: every number this chapter's generators write, checked against JPL Horizons.

PreviousDownloadable bodies NextPrecision