Data and the compute of discovery

In 2003 the Hubble Space Telescope’s Advanced Camera for Surveys stared at one small patch of the ecliptic for the faintest Kuiper belt objects it could reach. Gary Bernstein and collaborators processed those images with about 30,000 lines of code and several CPU-years on 2.4 GHz Pentium processors, spread over ten machines in Pennsylvania and eight in Arizona. Even so, they could not afford to stack the whole observing sequence along each trial orbit. They stacked subsets spanning at most about a day and linked the detections afterwards. The search found three new trans-Neptunian objects in about 0.02 square degrees, and its most important result was the faint objects it did not find (Bernstein et al. 2004, §§2-3).

Twenty-three years later Kevin Napier, Matthew Holman and collaborators searched the same photons again. Their heliostack method combined the images across the full fifteen-day baseline, used a few hundred NVIDIA H200 GPU-hours plus a few thousand CPU-hours, took less than a day of elapsed time, and reported two additional faint objects (Napier et al. 2026, §§3-4). Nothing about the telescope changed. What changed was which statistical experiment the data could afford.

This chapter is about the two ingredients of that experiment: the data that already exist, and the computation needed to test hypotheses against them. The first half is a working map of the survey archives a reader can search today, organized around one distinction: an archive of exposures is a different scientific resource from an archive of catalogued sources. The second half is a history of the compute of discovery, from human attention at a blink comparator to volunteer grids and GPU clusters. It argues for one discipline throughout: carry resources consumed, search performed and validated yield together, because any one of them alone is misleading.

Three kinds of archive

A survey archive can hold three different things. An archive of exposures keeps time-stamped pixels, with their calibrations, masks, noise estimates and PSF models. An archive of stationary-sky measurements keeps catalogues and coadds built on the assumption that the sky does not move. An archive of discoveries, such as the Minor Planet Center’s, keeps the objects someone has already found. Only the first supports a new search for a faint moving body. A deep galaxy catalogue can omit a readily detectable moving object. A public image viewer can show a coadd that has erased it. An exposure-footprint match can place the predicted position in a gap between detectors.

The discovery of 2017 OF201 shows all three archive types at work. Sihao Cheng, Jiaxuan Li and Eritas Yang searched sparsely sampled DECam Legacy Survey (DECaLS) exposures and found ten detections of one object in $g$, $r$ and $z$ between 2014 and 2018, at around $r=22.6$ and a distance near 85 au. They describe the search as related to the Dark Energy Survey’s sparse-linking approach, with details deferred to a later paper. With the resulting ephemeris, the Canadian Astronomy Data Centre’s Solar System Object Image Search and a suggestion from Mike Alexandersen, they found nine more detections in CFHT images from 2011-2012. After the announcement, amateur astronomer Sam Deen identified Sloan Digital Sky Survey detections from 2004 and 2009. A Magellan observation on 25 July 2025 brought the total to 24 observations over 20.84 years and an orbit with $a=830.1\pm0.8$ au and perihelion $q=44.9189\pm0.0012$ au (Cheng et al. 2025, v2, §2 and Table 1). The SDSS catalogue had classified its two detections as a star and a galaxy. The object was in the pixels all along. The catalogue simply had no category for it.

Chapter 6 places OF201 among the distant bodies, and Chapter 13 treats precovery as a technique. Here the point is about data. The most useful archive for a given search is not necessarily the largest or the deepest. Repeated exposures of one field in one night support coherent motion searches for faint TNOs. Sparse visits spread over years support orbit linking, provided the object is detected in enough single exposures. One old image can extend the arc of a known near-Earth object by years. These differences decide the algorithm.

What an archive must preserve

The exposure is the measurement

For exposure $e$, detector pixel $p$ and orbit parameters $\theta$, a useful image model is

where $B$ is the sky and instrumental background, the sum is the stationary sky (sources $s$ with fluxes $F_{se}$), $W_e$ is the exposure’s sky-to-pixel map, $P_e$ its spatially variable PSF, and $n$ the noise. The moving body enters through an exposure-integrated template,

To evaluate this, an archive must preserve the time system and exposure interval, the detector geometry and WCS, the observer’s position, the photometric calibration and the PSF. The trail length is $L=|\boldsymbol\mu|\Delta t$ for angular rate $\mu$, or $L/s$ pixels for pixel scale $s$, and a mid-exposure coordinate is adequate only when trailing and timing errors are negligible. For TESS or WISE the position must be computed from the spacecraft, not from a ground site. Pan-STARRS documentation warns that misreading the absent TIMESYS='TAI' keyword in some products produces timing errors of tens of seconds (MAST, PS1 archive).

The statistic needs weights, not just significance

Stack all retained pixels into a vector $d$ with covariance $C$ and a unit-amplitude template $a(\theta)$. The Gaussian matched-filter estimate and significance are

With diagonal noise and independent epochs this factorizes into per-exposure sufficient statistics, $\psi_e=\sum_p a_{ep}d_{ep}/\sigma_{ep}^2$ and $\phi_e=\sum_p a_{ep}^2/\sigma_{ep}^2$, with $\rho=\sum_e\psi_e/\sqrt{\sum_e\phi_e}$ (Chapter 1, Chapter 17). Masked pixels contribute zero weight, so the denominator changes as a trajectory crosses gaps, saturated stars and detector edges. An archive that keeps only a normalized significance image has thrown away what is needed to combine epochs correctly. Before resampling, a simplified variance is $\sigma^2\simeq B+S+R^2+D\Delta t$ (background, source shot noise, read noise squared, dark current), and resampling, reference subtraction and shared calibration add covariance that a per-pixel weight map does not fully represent.

Depth, cadence and completeness are different numbers

For $N$ equal, independent, background-limited images aligned on the right trajectory, the SNR grows as $\sqrt N$ and the limiting magnitude improves by

This is an ideal, not a property of a stationary coadd. A coadd built for galaxies rejects moving flux as an outlier, and unequal weights, correlated residuals and motion mismatch all reduce the gain. Published depths also use incompatible conventions: AB versus Vega magnitudes, $5\sigma$ versus $10\sigma$, point sources versus fiducial galaxies, coadds versus single exposures. Converting a $10\sigma$ depth to $5\sigma$ adds $2.5\log_{10}2=0.75$ mag in the ideal background-limited case (derived), but no conversion turns a catalogue depth into a moving-object completeness. The quantity a search needs is the selection function

for orbit $\theta$, absolute magnitude $H$ and colours $\mathbf c$, conditioned on $\mathcal E$, the actual list of exposures that exist and are usable, not the nominal survey footprint. The DES outer-Solar-System search models per-exposure completeness as a logistic function of magnitude and measures the full selection function with injected populations (Bernardinelli et al. 2022, §§2-3).

A tour of the searchable archives

The table gives a scale and the product that matters for each resource. The numbers describe specific products and are not equivalent moving-object limits. These holdings also overlap: DES and DECaLS are both DECam programmes, and Legacy Surveys processing reuses DES exposures, so downloading all three and counting their files as independent observations overstates both exposure time and sensitivity.

Sources: NOIRLab; Abbott et al. 2021; Legacy Surveys DR10; Aihara et al. 2022; HSCLA2020; PS1 News and PS1 archive; ZTF DR24; SkyMapper DR4; NEOWISE; TESS; Rubin DP2. All as of 16 October 2026.

The Dark Energy Camera family

The Dark Energy Camera reached first light on the 4 m Blanco telescope in September 2012. Its science focal plane has 62 CCDs of $2048\times4096$ pixels at $0.263''$ per pixel. The often quoted 570 megapixels includes guide and focus chips (Flaugher et al. 2015). Because DES, DECaLS, DEEP and many community programmes used the same camera, their archive allows cadence regimes to be compared without changing the instrument model. Start at the NOIRLab Astro Data Archive, whose official notebooks document JSON searches, authentication and retrieval by MD5 checksum. They also warn that default result limits can silently truncate a query, so partition by date and sky and reconcile counts (NOIRLab nat-nb).

Dense DECam sequences suit GPU stacking. DEEP’s strategy took roughly 120 s exposures over about four hours per field per half-night, aiming at about $m_R=26.2$ under typical conditions (Trujillo et al. 2023, §4). Sparse DES and DECaLS sequences favour orbit-space linking. The DES wide survey took roughly ten visits per filter over six years and deliberately avoided same-night repeats, so the usual nightly-tracklet assumption fails. Its single-epoch PSF depths at SNR 10 are $g,r,i,z,Y=23.57, 23.34, 22.78, 22.10, 20.69$ (Abbott et al. 2021, Table 1). The Legacy Surveys’ quoted design depths of $g=24.0$, $r=23.4$, $z=22.5$ are $5\sigma$ depths for a fiducial galaxy, not single-exposure point-source limits (Dey et al. 2019, §4). For a moving-object search, start from the survey-ccds manifest, which leads back to individual exposures; the Tractor catalogues model a stationary sky.

Deep and sharp: HSC and CFHT

Subaru’s Hyper Suprime-Cam data come in separate collections: the Strategic Program releases, the Legacy Archive of processed PI programmes, and raw data in SMOKA. PDR3 drew on 278 nights and reached Wide coadd depths of $g,r,i,z,y=26.5, 26.5, 26.2, 25.2, 24.4$ with median seeing of $0.61''$-$0.79''$ (Aihara et al. 2022, Table 1). These are stationary coadd depths. The April 2026 Legacy Archive release adds roughly 7,500 square degrees in at least one filter, from heterogeneous programmes whose actual cadence must be established field by field. Access requires registration and is restricted to non-commercial scientific and educational use (HSC PDR3 data access). Public scientific access is not a commercial data licence.

At the Canada-France-Hawaii Telescope, Stephen Gwyn developed MegaPipe to remove the barrier of downloading, calibrating and stacking large MegaCam datasets (Gwyn, MegaPipe). Gwyn, Norman Hill and JJ Kavelaars then built SSOIS, which searches observation footprints along an orbit rather than at a fixed position (CADC, SSOIS). SSOIS returns candidate images, not detectable flux: index coverage, detector gaps and the downstream archive must still be checked. The CFHT Legacy Survey’s full D1 $r$ stack has 640 constituent exposures totalling 204,300 s (CADC, CFHTLS inputs). For designing a motion search, the list of constituents is worth more than the headline stack depth.

Wide and repeated: Pan-STARRS, ZTF and SkyMapper

Pan-STARRS1 covers the sky north of declination $-30^\circ$. Its typical single-epoch $5\sigma$ depths are $g,r,i,z,y=22.0, 21.8, 21.5, 20.9, 19.7$, and its stationary stacks reach $23.3, 23.2, 23.1, 22.3, 21.3$ (MAST, PS1 archive). The difference of about 1.4 mag is not free depth for an unknown orbit. Pixel searches should use warps, individual exposures resampled onto sky cells, which carry interpolation covariance and can duplicate parts of one exposure in overlapping cells. The processing system has handled up to 4 TB per night and more than 4 PB of historical raw imagery (Magnier et al. 2016, §1), a different accounting from the public release. MAST opened a PostgreSQL-backed TAP service for DR2 on 29 September 2026, with schema changes that make migration more than a URL edit.

Rick White’s moving-target notebook for PS1 is a compact reproduction of a precovery. It finds candidate exposures with SSOIS, queries JPL Horizons at their exact times and site, retrieves cutouts and crossmatches epoch detections. For asteroid 174361 Rickwhite it finds 93 candidate observations, of which 30 cutouts are blank because the target fell in a gap or off the active detector (White, MAST). A third of the footprint matches held no pixels at the predicted position. That is a measured rate at which exposure-circle coverage overstates real coverage.

The Zwicky Transient Facility trades depth (median $5\sigma$ about $g=20.8$, $r=20.6$ in 30 s) for coverage and cadence (Masci et al. 2019). DR24 holds public exposures through 22 October 2025 and private-programme exposures released after an 18-month proprietary period, so the public archive is not one rectangular time interval. Its light curves are seeded by reference-image sources, so a moving body can be present in the pixels and absent from them. SkyMapper DR4 has an especially explicit schema that separates averaged dr4.master properties from per-image dr4.photometry detections. Detections rejected when building the averaged table can survive in the photometry table, and a master-only search discards exactly the unassociated measurements a moving-object search needs (SkyMapper DR4).

Rubin: the archive arriving

Rubin’s Data Preview 1, released 30 June 2025, contains seven roughly square-degree ComCam fields observed between 24 October and 11 December 2024. Its low-ecliptic-latitude field has 159 visits over 5 nights, and its Extended Chandra Deep Field South field has 855 visits over 21 nights, with average $r$-band visit limits of 24.19 and 24.15 (Rubin DP1). Those two cadence densities make good controlled workloads for search algorithms. Early DP2, released 27 July 2026, covers LSSTCam data from April 2025 to January 2026. It provides all catalogues, including measurements from visit and difference images, but only deep-coadd image products (Rubin DP2).

The reason pixels lag catalogues is plainly physical. Rubin’s October 2026 early-science plan says that insufficient network bandwidth between the US Data Facility and Google Cloud is delaying visit-image serving. Its forecasts place full DP2 and nightly processed images in November-December 2026 and the first annual data release in July-December 2028 (Guy et al. 2026, RTN-011, §§3.3, 4, Table 8). The same document notes that DP2’s Solar System tables associate detections with known MPC objects by position. An unmatched detection is still discovery material. Rubin data rights also restrict platform access to rights holders, with a two-year proprietary period for images and annual releases (Rubin data policy). Chapter 15 covers the alert stream.

Infrared and space: WISE and TESS

The final NEOWISE archive, released 14 November 2024, spans December 2013 to 1 August 2024. It holds 26,888,166 exposure image sets: each set is two $1016\times1016$ intensity images at $2.75''$ per pixel, with noise and mask images, from 7.7 s W1/W2 exposures. Confirmed moving-object tracklets contain 1,632,566 detections of 44,597 bodies (IRSA, NEOWISE). WISE’s resolution of about $6''$ in W1/W2 makes confusion an information limit that no archive size removes (Wright et al. 2010). Aaron Meisner and collaborators’ unTimely catalogue detects sources independently in about 16 epoch coadds per location from 2010 to 2020: 43.46 billion detections in 4.3 TB, built with 226,620 CPU-hours (Meisner et al. 2023, §§3, 7). It is a time-domain catalogue, not a linked moving-object catalogue.

TESS full-frame images trade pixel size for time density. Their cadence moved from 30 min to 10 min to 200 s across mission cycles, and the short onboard integrations are not archived individually. Jorge Martínez-Palomera, Amy Tuson, Christina Hedges and collaborators retrieved a moving corridor of pixels to precover the interstellar object 3I/ATLAS. It was invisible in single 200 s frames, but stacking frames from 7 May to 2 June 2025, weeks before its July discovery, gave 15 measurements above SNR 3 at TESS magnitudes from $20.9\pm0.29$ to $19.57\pm0.15$ (Martínez-Palomera et al. 2025, §§2-3). That is targeted precovery, not blind-search completeness, but it shows an exoplanet archive doing Solar System work after the fact (Chapter 23).

The archive search as a space-time join

A reproducible archive search has four stages before any expensive computation: discover metadata, validate exact coverage, stage the pixels once, and build a statistically specified search representation.

for each archive time/sky partition:
    rows = query_and_paginate(partition)
    save raw response, schema, release label, query, timestamp
    canonicalize exposure / detector / product / version identities
    deduplicate physical exposures (keep alternate reductions separately)

for each candidate orbit or requested sky-time region:
    candidates = intersect(indexed exposure footprints, uncertainty corridor)
    refine to detector polygons and active pixels at exposure time
    predict observer-specific position and exposure-integrated trail
    keep usable images with masks, variance, PSF, calibration, WCS

for each detector-time tile shared by many hypotheses:
    retrieve, checksum and decompress once
    model background; build weighted search statistics
    search or force-photometer many trajectories
    validate survivors on independent epochs and original pixels
    record provenance and injection-recovery outcomes

The costs differ by stage. A balanced spatial-temporal index over $E$ exposures costs about $O(E\log E)$ to build and $O(\log E+k)$ per corridor query returning $k$ candidates, against $O(E)$ for a scan (algorithmic estimates). After retrieval, decompression scales with bytes touched. PSF filtering of $N$ images of $P$ pixels costs $O(NPq)$ for a $q$-pixel kernel, or about $O(NP\log P)$ with FFTs, and accumulating $H$ trajectory hypotheses over $N$ epochs costs $O(HN)$ samples. Naive pair and triplet enumeration over $K$ catalogue detections costs $O(K^2)$ and $O(K^3)$. The hard large-scale problem is joining many unknown orbits to many irregularly placed exposures without enumerating their Cartesian product.

Working sets grow quickly. A hundred exposures of one $2048\times4096$ DECam CCD, stored as science, variance and mask at four bytes each, already take about 10.1 GB before PSFs, padding and temporary copies (estimate). Duplicated resampled and filtered copies can overflow an accelerator even though every file is modest. Streaming and tiled construction of the search statistics can matter more than peak arithmetic.

Known-object precovery and blind discovery need separate control logic. Precovery asks whether predicted flux exists in a small corridor. Blind discovery must find the corridor, account for the whole hypothesis family and show that chance alignments do not explain the result. Feeding a known ephemeris to a supposedly blind benchmark changes the experiment.

A short history of the compute of discovery

The compute of discovery is the cost of testing hypotheses against data, including the cost of deciding which apparent detections deserve another look. Its history is not one rising curve of operations per second. The scarce resource has changed several times.

Plates: the observer as the association engine

Clyde Tombaugh’s own account is unusually concrete (Tombaugh 1946; Chapter 4 tells the full story). The 13-inch Lawrence Lowell telescope recorded about $12^\circ\times14^\circ$ on a $14\times17$-inch plate, reaching about seventeenth magnitude in a standard one-hour exposure. A plate pair in Aquarius through Aries held about 50,000 star images each and took three days to examine thoroughly at the blink comparator. Plates in western Gemini held over 300,000, and the work rose roughly with image count. Tombaugh does not say how many hours made a working day, so converting this into hours per plate would invent a measurement.

Two choices of data, not of instrument, reduced the false-candidate burden. Observing near opposition separated the slow motion of distant bodies from asteroids near their stationary points. Taking three plates of each region within a week added 50% to the photographic work but rejected almost all emulsion defects without new observations. That is a sequential test: a cheap independent check before expensive follow-up. On the afternoon of 18 February 1930, about a quarter of the way through comparing plates from 23 and 29 January, Tombaugh found Pluto. He had examined about two million star images by then, and his continuing search eventually covered about 90 million. These are counts of attention, not independent statistical trials.

CCDs: computing on the observing night

David Rabinowitz’s 1991 paper on Spacewatch documents the moment computation began to change the observing night (Rabinowitz 1991). The Moving Object Detection Program ran at the 0.91 m telescope on a Solbourne workstation with three SPARC processors and 32 MB of memory, while an Intel 80386 handled CCD readout. It searched drift scans as they arrived, using trails for fast movers and position changes across three scans for slower ones, and put candidate cutouts in front of the observer. In nineteen trial nights in May and June 1990, covering 150 square degrees, it detected 304 asteroids to about $V=20.5$. The earlier discovery of 1989 UP had been reported quickly enough for follow-up about four hours later in Australia. The value of the computer was latency as much as throughput. The paper gives no integrated CPU-hour total, and none should be invented for it.

Deep CCD searches: telescope days become CPU-years

Bernstein’s Hubble search marked the next regime. Its fine search evaluated about $10^{14}$ points in position, motion and distance, and needed about $10^{16}$ pixel additions. Aren Heinze, Stanimir Metchev and Joseph Trollo made the work explicit in 2015 with a unit they called the vector-pixel: input images times pixels per image times trial motion vectors. Their two-night survey with the WIYN 0.9 m telescope used 126 and 130 images of $12{,}000^2$ pixels and 28,086 trial vectors, $1.035\times10^{15}$ vector-pixels in all. At $8.1\times10^{11}$ vector-pixels per hour on a 32-core workstation, the search took about fifty days and found 156 previously unknown asteroids and 59 known ones (Heinze et al. 2015, §§2.3.1, 5.1). The 64 GB of RAM could not hold the full images, so the data were split into four overlapping $6{,}000^2$ quadrants.

Their 2019 DECam search was more expensive and much more productive. It used about eighty 24-core node-days per pass and about 240 node-days including the false-positive and injection passes. It detected 3,234 distinct asteroids and was 95% complete at $R=25.0$ (Heinze et al. 2019, §4.2). The images were $32{,}000\times28{,}000$ pixels. At full precision the sequence would have needed about 690 GB. Internal $2\times2$ binning cut that to about 170 GB, and validated sixteen-bit storage brought it within a 128 GB node, with values restored to full precision for the stack itself. That is a documented case of precision reduction and footprint management enabling a search, years before tensor cores. The roughly threefold validation overhead is not waste: one pass searches for real objects, one measures false positives, one measures completeness with implanted sources. A benchmark that times only the first pass undercounts the scientific programme.

GPUs and the return to old photons

Peter Whidden and collaborators’ KBMOD replaced repeated image stacks with gathers of precomputed likelihood statistics, one GPU thread per trial trajectory (Whidden et al. 2019; Chapter 17, Chapter 40). Its measured coefficient, $2.4\times10^{-11}$ s per trajectory per image, implies about $1.5\times10^{14}$ trajectory-image contributions per hour on one GPU (derived), against $1.3\times10^{12}$ vector-pixels per hour per CPU node for Heinze’s clipped-median stacks. The two units measure different operators, so the ratio is not a fair speedup. Its HiTS application found 39 new Kuiper belt objects. The paper also shows where time goes after the scan. Half a million candidates passed through its light-curve filter in seventy seconds, and one extra bright-pixel mask removed about $4\times10^8$ false candidates in one field, saving hours of postprocessing.

DEEP turned this machinery into a survey. Its median one-CCD search of about a hundred images took 576 s on an NVIDIA A40, including loading, the scan, false-positive filtering and stamps, and the whole search ran for about 282 hours. Reverse-motion searches calibrated the artifact background, humans rejected about half the candidates that survived a CNN, and linking nightly detections across seasons yielded 110 multi-year TNO orbits, 105 of them apparently new (Smotherman et al. 2023, §§3-5).

Sparse linking at scale: DES

Pedro Bernardinelli, Gary Bernstein and the DES collaboration spent an estimated 15-20 million CPU-hours extracting outer-Solar-System orbits from a cosmology survey whose cadence had deliberately avoided the usual moving-object pattern (Bernardinelli et al. 2022; Chapter 12). The pipeline built a clean transient catalogue, then for each distance hypothesis transformed positions to reduce parallax curvature, enumerated compatible pairs, found third detections consistent with bound orbits, fitted and grew candidate orbits, and finally tested survivors against sub-threshold flux in exposures not used to find them. Candidates with fewer than nine detected nights had to show stacked significance above $1.2\sqrt{N_{\rm images}}$ in that independent stack.

The cost is output-sensitive. Catalogue density, distance bins and allowed motion determine how many intermediate combinations are generated, and a spatial index cannot remove the cost of emitting a huge valid intermediate set. The authors state that extending the search to closer objects was beyond their computational means, because the cost rises as a high inverse power of distance. They also found further reductions of the detection threshold computationally infeasible. Of the 817 objects found, 461 were reported for the first time, and the catalogue is characterized to about $r=23.8$ with injected populations.

Buying a bigger bank with volunteers

Pulsar and gravitational-wave searchers met the same problem earlier and bought their way through it with distributed volunteer computing (Chapter 36, Chapter 37). In 2010 Einstein@Home had about 100,000 hosts a week and an aggregate 0.25 PFLOP/s. Each roughly two-CPU-hour task searched 6,661 circular binary-orbit templates and returned its hundred best candidates, and code and algorithm optimization had sped the search up about sevenfold. It found PSR J2007+2722 in archival Arecibo data (Knispel et al. 2010). The 2013 Parkes reanalysis used about 17,000 CPU core-years and found 24 new pulsars. Building its 12,140-template orbital bank alone cost about 200,000 CPU-hours, and GPUs completed a task ten to twenty times faster than CPUs (Knispel et al. 2013, §3).

The 2016 all-sky search for continuous gravitational waves in LIGO S6 data evaluated $6.3\times10^{16}$ templates in 12.7 million workunits, each sized to occupy a volunteer computer for about six hours and to return only its best 3,000 candidates. No signal survived, and the search set a 90%-confidence strain upper limit of $5.5\times10^{-25}$ in its best band (LIGO/Virgo 2016). Multiplying workunits by six hours gives about 76 million nominal host-hours (estimate; it ignores redundant validation copies and host heterogeneity). A null search like this has no cost per discovery at all. Its value is the excluded amplitude and the parameter space covered.

The 2020s: geometry, data movement and catalogues

Three recent works show the cost moving away from arithmetic. Heliostack shares orbital geometry across whole images. For each of 1,148,817 asserted initial conditions it propagates image-corner rays to a common epoch, forms a perspective warp, and warp-adds signal and variance onto a barycentric canvas (Chapter 20). Its 1,118 images produced 109,383 candidate records, 533 after deduplication by trajectory, and 145 after orbit refinement and a final threshold of 8.8: 141 implanted sources and four apparently real objects. The bank was restricted to cold-classical orbits ($a=42.4$-$47.4$ au, $e<0.2$, inclination below $5^\circ$, distance 38-51 au) at a tracking tolerance of $0.05''$.

Nathan Golovich and colleagues at Lawrence Livermore built survey-wide nonlinear digital tracking for the Lassen supercomputer (Golovich et al. 2025). Their development archive was about 400 TB of compressed ZTF difference images from March 2018 to February 2024, about 56 million quadrant files. Images are partitioned across nodes, GPUs compute orbit-image intersections, CPUs retrieve the relevant pixel patches from node-local SSDs, and evidence is combined across nodes. Tiling images into $32\times32$ pixel blocks cut SSD page traffic for a typical $10\times10$ patch by about five times, a compute gain obtained purely from data layout. Their weak-scaling test isolates what limits the system:

Geometry stays flat as nodes are added. Retrieval quadruples, because the slowest of 128 ranks receives an increasingly unfavourable batch: the maximum number of intersections assigned to one rank in an 8,000-orbit batch grows from about thirty to ninety-one. Adding accelerators does not repair that tail.

Steven Stetzler and colleagues took the opposite approach and stacked low-significance detection catalogues rather than pixels (Stetzler et al. 2025; Chapter 21). At a catalogue threshold of two they reached a 50% completeness magnitude of $25.21\pm0.05$ in five CPU-minutes and 15.4 GB, at least 30.8 times faster than their CPU image-stacking reference (2.54 core-hours, 17.5 GB) and about 0.25 mag shallower than a separate KBMOD run at $25.47\pm0.01$. At threshold one they reached $25.87\pm0.21$, deeper than that KBMOD run, but with 39.5 CPU-minutes and 190.8 GB, and without demonstrated false-alarm parity. This is a measured trade surface between depth, time and memory, not a thirtyfold speedup over GPU KBMOD at equal sensitivity.

Why deeper searches cost more

Let $E$ be the number of exposures, $P$ the number of starting pixels, $K$ the number of trial motions, $T$ the elapsed baseline and $\epsilon$ the tolerated sky-position error. A direct linear search costs

where $A_v$ is the area of velocity space searched. Trial velocities must be spaced so that the end-of-baseline position error stays below $\epsilon$, which gives the $T^2$. With exposures at a fixed cadence, $E$ also grows with $T$, so the work grows roughly as $T^3$ before any acceleration terms become measurable. For nonlinear searches, more parameters become resolvable as the arc lengthens, and each adds a dimension. There is no universal exponent. Bernstein’s short-arc Hubble grid scaled about as $p^{-5}T^3$ for spatial grid spacing $p$, and the choice between $0.030''$ and $0.050''$ grids was a computational decision with sensitivity consequences. Heliostack reports that tightening its tolerance from $0.05''$ to $0.03''$ would need about four times the computing, and that widening its bank to the general Kuiper belt with inclinations below $60^\circ$ would need about 300 times more initial conditions (Napier et al. 2026, §3.2).

A bigger bank also raises the bar for significance. For $N_{\rm eff}$ effectively independent standard-normal trials, the expected number of false excursions above $\nu_\ast$ is

At $N_{\rm eff}=2\times10^{14}$ a $7\sigma$ threshold gives about 256 excursions under this idealized model (estimate). Heliostack observed 368 non-injected candidates after deduplication at its initial threshold, broadly that scale. Overlapping templates make $N_{\rm eff}$ smaller than the executed count. Real artifacts make the tail heavier.

The scan is one term in a larger bill:

The terms have different units until someone converts them, and candidate handling can dominate even when the exhaustive scan is cheap. KBMOD’s $4\times10^8$ masked false candidates and DEEP’s human rejection of half its post-CNN candidates are measurements of the $N_{\rm cand}$ term. ATLAS publishes a per-image CPU breakdown in which subtraction and difference-source detection cost 1,350 CPU-seconds and the moving-object client 120 (Tonry et al. 2018, Table 3; Chapter 14). The search kernel is rarely the whole cost.

The numbers, side by side

The figure is deliberately a scatter of labelled points, not a curve. Each point is in its source’s own unit, and the scopes differ: a benchmark kernel, a search stage, a whole survey, a catalogue build. The vertical spread of nine orders of magnitude does not measure progress in efficiency. It reflects the different questions asked: a 180-second kernel benchmark sits beside seventeen thousand core-years of volunteer pulsar searching. What the figure does show is that the resources spent on discovery span from a few minutes on one GPU to the largest volunteer grids, and that both GPU entries are small in absolute terms because they bought dense searches of compact fields.

Dividing by yield helps only with the scope held fixed:

All derived from the published totals cited above. Note what the table cannot say. DES found its objects in a survey that was never designed for them and characterized the whole selection function, while DEEP observed specifically for moving bodies. The costs differ by four orders of magnitude, and neither is the cheaper way to answer the other’s question.

What the numbers cannot tell us

The Hubble repeat is not a clean hardware benchmark. The two analyses differ in bank, temporal combination, tracking tolerance, calibration version and thresholds. Heliostack’s magnitudes for common objects are about 0.2 mag brighter than the old values, which the authors attribute to possible zeropoint or calibration-software differences without fully resolving it. So the nominal completeness midpoints, 29.21 against 29.17, do not show a 0.04 mag gain. The authors estimate an effective gain of about 0.2 mag: usable integration rises from roughly 22,000 to 38,000 s, partly offset by a stricter threshold (Napier et al. 2026, §3.5). The two new objects carry temporary labels, not MPC designations, and their short arcs do not securely classify them.

The denominator is often the main uncertainty. “CPU-hour” may mean allocated core-hours, process CPU time, or an equivalent workload on mixed hardware. “GPU-hour” may omit host processing and data staging. Pan-STARRS reprocessing used Los Alamos’s 3,088-node Mustang system, but that does not mean the whole machine was devoted to asteroid discovery, and charging its capacity to a discovery count would manufacture a cost (Magnier et al. 2016). Multiplying THOR’s 23 nodes of 28 cores by its eighteen elapsed hours gives an allocation envelope, not consumed core-hours, and its individual jobs ranged from ten minutes to sixteen hours (Moeyens et al. 2021, §4.3). This chapter keeps source wording rather than resolving such ambiguities by assumption.

Dollars and energy are not recoverable. Capital, amortization, electricity, allocation policy, negotiated cloud prices and researcher time are almost never reported. Heinze’s 2015 paper sketches a hypothetical twelve-workstation system costing about 60,000 dollars, but that is a design scenario, not the cost of its discoveries (Heinze et al. 2015, §2.3.3).

Artifacts can erase arithmetic gains. Ground-based difference images contain subtraction residuals and correlated defects. DEEP’s reversed searches, raised thresholds, CNN and human vetting measure that problem directly. Hubble’s stable PSF let heliostack reach a short final list without visual inspection or machine learning, and its authors warn that such purity may not carry over to variable-quality ground data.

Release status can be the binding constraint. Rubin’s visit pixels wait on network bandwidth, Pan-STARRS’s interface changed schema in September 2026, ZTF replaced a metadata export with a weekly pointing-history file, and access rights differ archive by archive: HSC’s non-commercial restriction, Rubin’s rights holders, proprietary periods at CFHT and ZTF. None of these is an arithmetic problem, and a faster GPU does not remove any of them.

Where the room is

The physical limits are photons not collected, onboard coaddition that cannot be undone (TESS), saturation, confusion at coarse resolution (WISE), and sky never covered. Coverage and cadence also couple to orbital elements through Galactic-plane holes, hemisphere boundaries, seasonal filters and chip gaps. Everything else in this chapter is engineering: file latency, redundant processing, catalogue schemas, approximate footprints, avoidable resampling, poorly batched orbit propagation, load imbalance. Those can be reduced. Proprietary access is policy.

Join space and time before accelerating anything. The first opportunity is a compact detector-footprint and time index that keeps exact coverage and groups hypotheses by the pixels they share, so each detector-time tile is retrieved, decompressed and background-modelled once for every trajectory that needs it. Schedule by shared data, not by object. Cache keys must include reduction version, mask definition, calibration and coordinate frame, or the reuse is scientifically invalid. The LLNL measurements point the same way: larger asynchronous batches, GPU geometry overlapped with CPU reads, and scheduling by expected intersection count to remove the retrieval tail.

Search the right intermediate representation. A common weighted $(\psi,\phi)$ representation separates expensive per-image work from trajectory summation and travels well between archives. Hierarchies can reject impossible orbit regions on compact nightly summaries and refine survivors on original pixels, provided they keep the evidence signed (Chapter 21). The archives where such reuse pays are those with many overlapping visits along a trajectory: Rubin deep drilling fields, DEEP, HSC and CFHT sequences, Hubble and JWST stares. Many nominally enormous archives have little overlap along any one track. Tensor-core reformulations need hundreds of overlapping exposures per output to pay (Chapter 40).

Make sparse linking output-sensitive. DES’s $10^{13}$ pairs show a regime where image-stacking arithmetic is not the problem at all. Better orbit-space partitioning, admissible geometric bounds and sparse joins could reduce candidate expansion, which is what made closer distances and lower thresholds unaffordable. A hybrid that seeds a broad hypothesis set from permissive catalogue detections and then tests it with signed pixel evidence from held-out exposures combines the cheapness of catalogues with the information in pixels. Population injections should measure the loss at each gate.

Benchmark the experiment. The most valuable missing artifact in this field is a frozen benchmark: fixed input pixels and exposure manifest, PSF and noise treatment, orbital coverage, tolerated mismatch, injected population and candidate budget. It would report completeness by magnitude and orbit, retained false candidates, all-stage wall time, CPU and GPU resources, peak memory and archive bytes read. With that, a claimed order-of-magnitude gain could be checked against the same recovery curve. The fragments exist in the papers cited here, but they do not yet fill one table. Chapter 42 returns to what such a benchmark would make possible. A tenfold increase in usable independent exposures buys only 1.25 magnitudes. A tenfold saving in compute at fixed exposures buys no depth at all unless it is spent, explicitly, on a bigger search.

References

1. Abbott, T. M. C. et al. (DES Collaboration) (2021). The Dark Energy Survey Data Release 2. ApJS. arXiv:2101.05765
2. Aihara, H. et al. (2022). Third data release of the Hyper Suprime-Cam Subaru Strategic Program. PASJ. arXiv:2108.13045
3. Bernardinelli, P. H., Bernstein, G. M., Sako, M., Yanny, B. et al. (2022). A search of the full six years of the Dark Energy Survey for outer Solar System objects. ApJS. arXiv:2109.03758
4. Bernstein, G. M., Trilling, D. E., Allen, R. L., Brown, M. E., Holman, M., Malhotra, R. (2004). The size distribution of trans-Neptunian bodies. Astronomical Journal. arXiv:astro-ph/0308467
5. Canadian Astronomy Data Centre (2026). Solar System Object Image Search documentation. cadc-ccda.hia-iha.nrc-cnrc.gc.ca
6. Canadian Astronomy Data Centre (2021). CFHTLS MegaPipe input images and limiting magnitudes. cadc-ccda.hia-iha.nrc-cnrc.gc.ca
7. Cheng, S., Li, J., Yang, E. (2025; v2 2026). Discovery of a dwarf planet candidate in an extremely wide orbit: 2017 OF201. arXiv preprint. arXiv:2505.15806
8. Dey, A. et al. (2019). Overview of the DESI Legacy Imaging Surveys. Astronomical Journal. arXiv:1804.08657
9. DESI Legacy Imaging Surveys (2026). Legacy Surveys DR10 data release description. legacysurvey.org
10. Flaugher, B. et al. (2015). The Dark Energy Camera. Astronomical Journal. arXiv:1504.02900
11. Golovich, N., Steil, T., Geringer-Sameth, A., Iwabuchi, K., Dozier, R., Pearce, R. (2025). Survey-wide asteroid discovery with a high-performance computing enabled non-linear digital tracking framework. Astronomy and Computing 53, 100987. arXiv:2503.08854
12. Guy, L. P. et al. / Rubin Observatory (2026). Rubin Observatory plans for an early science program, RTN-011 v9.1. rtn-011.lsst.io
13. Gwyn, S. D. J. / CADC (2021). Introduction to MegaPipe. cadc-ccda.hia-iha.nrc-cnrc.gc.ca
14. Heinze, A. N., Metchev, S., Trollo, J. (2015). Digital tracking observations can discover asteroids ten times fainter than conventional searches. arXiv preprint. arXiv:1508.01599
15. Heinze, A. N., Trollo, J., Metchev, S. (2019). The flux distribution and sky density of 25th magnitude main belt asteroids. Astronomical Journal. arXiv:1910.13015
16. HSC-SSP (2026). HSC PDR3 data access. hsc-release.mtk.nao.ac.jp
17. Knispel, B., Allen, B., Cordes, J. M. et al. (2010). Pulsar discovery by global volunteer computing. Science. arXiv:1008.2172
18. Knispel, B., Eatough, R. P., Kim, H., Keane, E. F., Allen, B. et al. (2013). Einstein@Home discovery of 24 pulsars in the Parkes Multi-beam Pulsar Survey. ApJ. arXiv:1302.0467
19. LIGO Scientific Collaboration and Virgo Collaboration (2016). Results of the deepest all-sky survey for continuous gravitational waves on LIGO S6 data running on the Einstein@Home volunteer distributed computing project. arXiv preprint. arXiv:1606.09619
20. Magnier, E. A., Chambers, K. C., Flewelling, H. A. et al. (2016). The Pan-STARRS data processing system. ApJS. arXiv:1612.05240
21. Martínez-Palomera, J. et al. (2025). Pre-discovery TESS observations of interstellar object 3I/ATLAS. arXiv preprint. arXiv:2508.02499
22. Masci, F. J. et al. (2019). The Zwicky Transient Facility: data processing, products, and archive. PASP. arXiv:1902.01872
23. MAST/STScI (2026). Pan-STARRS1 data archive home page. outerspace.stsci.edu
24. MAST/STScI (2026). PS1 News. outerspace.stsci.edu
25. Meisner, A. M. et al. (2023). unTimely: a full-sky, time-domain unWISE catalog. Astronomical Journal. arXiv:2209.14327
26. Moeyens, J., Jurić, M., Ford, J. et al. (2021). THOR: an algorithm for cadence-independent asteroid discovery. Astronomical Journal. arXiv:2105.01056
27. Napier, K. J., Holman, M. J., Lin, H.-W., Gerdes, D. W., Ruch, T. R. (2026). heliostack: a novel approach to minor planet discovery. Planetary Science Journal (accepted). arXiv:2604.16731
28. NASA/IPAC IRSA (2024). NEOWISE final data release. irsa.ipac.caltech.edu
29. NASA TESS Science Support Center (2026). TESS data products information. heasarc.gsfc.nasa.gov
30. NAOJ (2026). Hyper Suprime-Cam Legacy Archive: HSCLA2020. hscla.mtk.nao.ac.jp
31. NOIRLab (2026). About the NOIRLab Astro Data Archive. astroarchive.noirlab.edu
32. NOIRLab (2020). NOIRLab Astro Data Archive API notebooks. github.com/NOAO/nat-nb
33. Rabinowitz, D. L. (1991). Detection of Earth-approaching asteroids in near real time. Astronomical Journal 101, 1518. ADS
34. Rubin Observatory (2026). Data Preview 1: observations. dp1.lsst.io
35. Rubin Observatory (2026). Data Preview 2. doi:10.71929/rubin/3382528
36. Rubin Observatory (2026). Rubin data policy. rubinobservatory.org
37. SkyMapper / ANU (2024). SkyMapper Southern Sky Survey Data Release 4. doi:10.25914/5M47-S621
38. Smotherman, H. et al. (2023). The DECam Ecliptic Exploration Project (DEEP) VI: first multi-year observations of trans-Neptunian objects. arXiv preprint. arXiv:2310.03678
39. Stetzler, S., Jurić, M., Bernardinelli, P. H. et al. (2025). An efficient shift-and-stack algorithm applied to detection catalogs. Astronomical Journal. arXiv:2509.26279
40. Tombaugh, C. W. (1946). The search for the ninth planet, Pluto. Leaflets of the Astronomical Society of the Pacific 5, 73-80. ADS
41. Tonry, J. L., Denneau, L., Heinze, A. N. et al. (2018). ATLAS: a high-cadence all-sky survey system. PASP. arXiv:1802.00879
42. Trujillo, C. A. et al. (2023). The DECam Ecliptic Exploration Project (DEEP) II: observational strategy and design. arXiv preprint. arXiv:2310.19864
43. White, R. / MAST (2024). How to search for moving targets in PS1 images and catalogs. outerspace.stsci.edu
44. Whidden, P. J., Kalmbach, J. B., Connolly, A. J. et al. (2019). Fast algorithms for slow moving asteroids: constraints on the distribution of Kuiper Belt objects. Astronomical Journal. arXiv:1901.02492
45. Wright, E. L. et al. (2010). The Wide-field Infrared Survey Explorer (WISE): mission description and initial on-orbit performance. Astronomical Journal. arXiv:1008.0031
46. ZTF/IPAC (2026). ZTF Public Data Release 24 release notes. irsa.ipac.caltech.edu