Quantifying Uncertainties on the Tip of the Red Giant Branch Method

Barry F. Madore; Wendy L. Freedman; Kayla A. Owens; In Sung Jang

doi:10.3847/1538-3881/acd3f3

Quantifying Uncertainties on the Tip of the Red Giant Branch Method

Madore, Barry F.; Freedman, Wendy L.; Owens, Kayla A.; Jang, In Sung 2023-07-01 00:00:00 We present an extensive grid of numerical simulations quantifying the uncertainties in measurements of the tip of the red giant branch (TRGB). These simulations incorporate a luminosity function composed of 2 mag of red giant branch (RGB) stars leading up to the tip, with asymptotic giant branch (AGB) stars contributing exclusively to the luminosity function for at least a magnitude above the RGB tip. We quantify the sensitivity of the TRGB detection and measurement to three important error sources: (1) the sample size of stars near the tip, (2) the photometric measurement uncertainties at the tip, and (3) the degree of self-crowding of the RGB population. The self- crowding creates a population of supra-TRGB stars due to the blending of one or more RGB stars just below the tip. This last population is ultimately difﬁcult, although still possible, to disentangle from true AGB stars. In the analysis given here, the precepts and general methodology as used in the Chicago-Carnegie Hubble Program (CCHP) have been followed. However, in the appendix, we introduce and test a set of new tip detection kernels, which internally incorporate self-consistent smoothing. These are generalizations of the two-step model used by the CCHP (smoothing followed by Sobel-ﬁlter tip detection), where the new kernels are based on successive binomial-coefﬁcient approximations to the derivative-of-a-Gaussian edge-detector, as is commonly used in modern digital image processing. Uniﬁed Astronomy Thesaurus concepts: Red giant stars (1372); Distance indicators (394) 1. Introduction nearly a full magnitude closer at (m − M) = 24.25 ± 0.15 mag. The TRGB distance fell in the mid-range, at (m − M) = Over a century ago, Shapley (1918, 1919, 1930) used blue- 24.8 ± 0.2 mag, right between the other two extremes. How- sensitive photographic plates to measure (by eye) the mean ever, not all of the early cross-comparisons of TRGB and apparent magnitudes of the 25 brightest stars in galactic Cepheid distance scales were in conﬂict. For example, globular clusters (his Table 1, 1919), in order to go on to Freedman (1988a) used the ﬁrst CCD camera available on (incorrectly) build a case for his version of an Island Universe the CFHT and measure the TRGB in the halo of the Local cosmology (see Berendzen et al. 1976). With the availability of Group dwarf irregular galaxy, IC 1613. She found a true newly developed, red-sensitive photographic plates, Walter distance modulus of (m − M) = 24.2 mag, which did agree the Baade (1944) serendipitously resolved the brightest red giant Cepheid-based distance modulus of (m − M) = 24.3 mag stars (which, to his surprise, suddenly appeared at approxi- (Freedman 1988b). mately the same red-band magnitudes) in several dwarf The TRGB method ﬁnally came of age with the publication elliptical companions galaxies to the Andromeda galaxy, of two papers: the ﬁrst was the calibration paper by Da Costa & M31. That unanticipated discovery precipitated a revision in Armandroff (1990) who were inspired to undertake an I-band the size and age of the universe by a factor of 2. Four decades CCD survey of a sample of 8 southern Milky Way globular later, and armed with some of the ﬁrst available panoramic clusters. In doing so, they demonstrated that, while the mean linear charge-coupled devices (CCDs), Mould et al. (1984) colors of the giant branches were rank-ordered by the mean revisited one of Baade’s original dwarf galaxies, NGC 205. metallicities of the parent globular clusters (as previously They produced full color–magnitude diagrams (CMDs) known to Frogel et al. 1983 from pioneering their studies of revealing a broad swath of red giant branch (RGB) stars all RGB stars in globular clusters in the near infrared), the of which cumulatively deﬁned a constant I-magnitude plateau brightest of those RGB stars had a remarkably stable absolute in the CMD, later to be named the tip of the red giant branch, or magnitude, in the I band, independent of color. The second simply known by its initialism, the TRGB. They also had paper was that of Lee et al. (1993). It laid out, in one place, earlier observed NGC 147 (Mould et al. 1983) ﬁnding the same most of the key issues concerning systematics involving feature. But perhaps more interestingly, they (Mould & reddening, metallicity, star formation history, and host galaxy Kristian 1986) observed TRGB stars in the halo of the Local type, etc. It also introduced the widely adopted Sobel ﬁlter for Group spiral galaxy, M33. By good fortune, at about the same precisely deriving the magnitude at which the discontinuity in time, M33 had been the subject of two different investigations the RGB luminosity function (LF) occurs, as well as its into Cepheid distance moduli to this galaxy: one by Sandage & uncertainty, while exploring a range of smoothing kernels. This Carlson (1983) coming in high, with a value of was carried out in the context of anticipating a refurbished (m − M) = 25.23 mag; and another by Madore et al. (1985) Hubble Space Telescope, and applying the TRGB method widely to the extragalactic distance scale. The authors Original content from this work may be used under the terms demonstrated its ground-based application to 10 galaxies of the Creative Commons Attribution 4.0 licence. Any further spanning a wide range of Hubble types, metallicities, and distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI. absolute magnitudes, and found overall consistency in the 1 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. TRGB, Cepheid, and RR Lyrae distance scales at the level of populated individual galactic globular clusters, or very small 0.1 mag. The success of the TRGB method might be measured ﬁelds of view in the halos of very nearby galaxies, say). The by its subsequent adoption: over the intervening three decades, ﬁeld has matured, the demand for higher precision has more than 500 TRGB distances to nearby galaxies have been prevailed, and the numbers of stars measured in extragalactic published (for instance, NED lists over 900 references to halo ﬁelds have gone into the thousands, while at the same time TRGB distance determinations to 302 distinct galaxies; and other sources of uncertainty in determining the precision of the EDD lists 588 galaxies that they have derived uniformly TRGB have become clear. We feel that it is time now to processed TRGB distances ). And recently, the TRGB method explore parameter space a bit more thoroughly. In the has been extended to the calibration of Type Ia supernovae and following, we consider, in turn, a total of three independent, determination of the Hubble constant (Freedman et al. 2019; major sources of uncertainty: Freedman 2021). (1) The formal way in which the statistical uncertainty in the For theoretical discussions of the evolution of stars up to and tip magnitude can be quantiﬁed, speciﬁcally in terms of including degenerate helium core ﬂash, we recommend that its sensitivity to numbers of stars at the tip, and its readers consult the monographs by Cassisi & Salaris (2013), independent sensitivity to individual photometric errors Salaris & Cassisi (2005), and Lamers & Levesque (2017). For of those same tip-deﬁning stars. updated discussions of modeling, with special reference to (2) The effects of having an asymptotic giant branch (AGB) near- and mid-infrared applications of the TRGB method, see population of stars contributing to the 1 mag interval Serenelli et al. (2017), McQuinn et al. (2019), and Durbin directly above the TRGB. et al. (2020). (3) And ﬁnally, the explicit modeling of the mutual (line-of- sight) crowding of all stars along the RGB, and the 2. Motivation inevitable production of a new, but totally spurious, population of (crowded) stars, systematically brighter A quick census of the published determinations of the than the TRGB. apparent magnitude of the TRGB in even the nearest of galaxies (NED-D 2022 August version) immediately reveals a We use a modiﬁed Sobel edge-detection ﬁlter (see wide range of quoted uncertainties. The published errors, for Appendix B) for measurement of the TRGB, which is largely the tip measurement in a given galaxy, can vary by as much as consistent with our GLOESS-smoothed, Sobel-ﬁltered analysis a factor of 10; as in the case of M31 (0.05–0.57 mag used in the CarnegieChicago Hubble Program (Hatt et al. 2017; uncertainties quoted) and M33 (0.03–0.30 mag); but more Hoyt et al. 2018; Jang et al. 2018; Madore et al. 2018).In typically, they range over a factor of 3–6 as, for example, in the Appendix B, we derive and tabulate a complete series of new published values for the nearest galaxies: the LMC digital ﬁlters that are derived from successive discrete (0.04–0.25 mag), IC 1613 (0.05–0.20 mag), and NGC 6822 approximations of the ﬁrst derivative of a Gaussian (DoG), (0.06–0.19 mag). On the other hand, some of the reported using the binomial theorem as the gradient detector. We also statistical uncertainties on the tip determination can go as low adopt the weighted (noise-suppression) versions of these as 0.01 mag (e.g., Lee & Jang 2012 for Messier 101 (M101);or kernels as ﬁrst introduced and applied to a simple Sobel ﬁlter even smaller than that in the case of Conn et al. 2011 for in Madore et al. (2009) and much later adopted and utilized by Andromedas I and II). In an appendix to Cioni et al. (2000), Gorski et al. (2018). those authors rightly note that many of the methods used, (counterintuitively) do not in any way scale with population 3. The Underlying Model size of stars detected and measured at the tip. They should. For The basic model adopted here for the intrinsic LF, above and a given photometric error, population size certainly needs to be below the TRGB, now consists of three distinct input a part of the calculation of the statistical uncertainty on the populations: (a) a RGB population with a power-law increase mean of the TRGB distance. Upon closer examination of any in numbers with increasing (fainter) magnitudes, (b) a bright- given paper, it is not always clear what exactly the source of the end truncation/discontinuity of the RGB LF, deﬁning the tip, quoted uncertainty is or even how it was actually calculated. In and (c) an AGB population, stretching at least 1 mag above and this paper, we attempt to bring some clarity to the situation. brighter than the TRGB. We model the LF from 1 mag above In earlier papers (Madore & Freedman 1995; Madore to 2 mag below the TRGB (but note that only the ﬁrst et al. 2009), we presented computer simulations of the TRGB magnitude below the tip is shown in the ﬁgures) assuming a ﬂat in its use as an extragalactic distance indicator. In the ﬁrst LF for the AGB down to the TRGB, at which point there is a paper, there is an often quoted and paraphrased conclusion that discontinuous offset to the RGB population. The RGB then “at least 100 stars in the ﬁrst magnitude interval below the tip assumes a steeply rising LF of the form log[( Nm)]= are needed to secure a distance modulus to better than ±0.1 0.3´-[] II +a. In these ﬁrst simulations, the relative TRGB mag.” At that time, the method was still in its infancy, and RGB-to-AGB normalization is six to one, such that there are 17 small number statistics were a major concern (especially when AGB stars in total in the 1 mag interval seen above the TRGB, the early focus was on applying the method to sparsely for every 100 RGB stars in the 1 mag interval fainter than (i.e., below) the tip. For the purpose of this simulation, the AGB LF https://ned.ipac.caltech.edu https://edd.ifa.hawaii.edu/dsecond.php That said, the errors presented in NED are in no way homogenized. NED The referee has argued that a variety of shapes to the AGB luminosity is presents the data as published, and in many cases, the original authors make no apparent in published CMDs including data above the TRGB, and that a ﬂat distinction between statistical and systematic error, or combinations of the two. AGB LF may not be representative. We agree with that statement, but as shown However, see Menendez et al. (2002) and/or Makarov et al. (2006) for in Appendix D, the shape of the AGB LF, be it falling rising or ﬂat, has no extensive discussions speciﬁcally concerning the maximum-likelihood techni- impact on the ability of the Sobel ﬁlter to detect the TRGB in an unbiased que and its error sensitivity to photometry and sample size. manner. 2 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. magnitudes) of TRGB. The lower panel shows the result of running a Sobel ﬁlter [−1, 0, +1] across the binned LF. The output of the Sobel ﬁlter is the discretely sample ﬁrst derivative of the function being sampled. Moving from left to right, the output of the Sobel ﬁlter is constant, as expected, given the constant slope of the input AGB luminosity, i.e., at the ﬁrst position, the output of the Sobel function is [−1 × (N − 3ò) + 0 × 2ò + 1 × (N − 1ò)=] 2ò, where ò is the width of the binning. At the second position, the output is [−1 × (N − 2ò) + 0 × (N − ò) + 1 × N)=] again (the slope of the pure AGB) 2ò. These ﬁrst steps do not sample the discontinuity and therefore contain no information about its position or presence. At step No. 3, the right-most element [+1] is the ﬁrst to sample the discontinuity and reports a increased value of the ﬁlter’s output, [−1 × (N + ò) + 0 × N + 1 × (6/2)N=] 2N − ò. The next step over continues to report larger values of its response-function, where the [+1] element now sees the undiluted height of the RGB LF 6N, and differences that against the response of the left-most element [−1] of the Sobel ﬁlter contributing a value of −N, with the central element of the Sobel ﬁlter always reporting a null value regardless of the function’s value. The output is [−1 × N + 0 × (6/2)N + 1 × 6N=] +5N. The central element simply keeps track of the bin around which the derivative is being measured and reported. Moving the ﬁlter one more bin to the right reports a value of [−1 × (6/2)N + 0 × 6N + 1 × (6N+δ)=] +3N + δ. One step more away from the discontinuity gives [−1 × 6N + 0 × (6N+δ) +1 × (6N +2δ)=] +2δ, the slope of the RGB. All subsequent steps to the right continue to report the constant slope of +2δ. The Figure 1. Magniﬁed view of the idealized toy model of the RGB + AGB luminosity function, centered on the discontinuity in the RGB luminosity maximum value of 5N for the response-function is found at function at the TRGB. Solid yellow segments, from left to right, show the AGB step No. 4 and marks the magnitude at which the TRGB is to LF, the discontinuity, and the RGB LF. The blue histogram is a binned version be found. of straight lines used as digital input to the differencing (Sobel) kernel: [−1, 0, +1]. The digital output of the Sobel response-function is shown in red histogram form in the lower portion of the ﬁgure. The maximum of the Sobel 4. A Few Preliminaries ﬁlter marks the position of the discontinuity. See text for a step-by-step description of the tip detection. It is worth making explicit what exactly the criteria are for a successful experiment to be run, which aims for a detection and is assumed to be ﬂat in the 1 mag interval above the tip and measurement of the position of the discontinuity marking the zero elsewhere. References to the literature justifying the values TRGB in magnitude space. It is then also important to list the for the parameters alluded to above are to be found in the ﬁrst real-world parameters over which we have some control in paragraph of Section 5.1.1. optimally undertaking the observations and subsequently Here we ﬁrst examine an idealization in the form of a toy analyzing the results. model that captures the essential ingredients of the detailed Generally speaking, there are two obvious performance simulations that follow, and try to emphasize how the various indices in TRGB edge-detection that we are concerned with components contribute (or not) to the determination of the here: accuracy and precision. However, the latter (which can magnitude and location of the TRGB discontinuity. The toy also be classiﬁed as bias) can be further broken down into (a) model is shown in Figure 1, an LF centered on the TRGB. This false-positive detections of the TRGB, (b) nondetections, and a plot of logarithm of numbers of stars per magnitude bin as a (c) systematic bias attributed to the edge-detector itself. Each of function of magnitude. The LF is composed of an AGB these are discussed in turn, below. And in the subsection population, represented by a dispersionless straight line sloping following this, we discuss what control we have, at the upward from left to right, stopping one bin short of the location observational design level, in mitigating each of these kinds of of the discontinuity deﬁning the TRGB. The number of AGB errors. stars in that ﬁnal bin is N. One bin beyond that magnitude, the LF is deﬁned by RGB stars whose slope is independent of, and 4.1. Accuracy different from, the AGB slope. The RGB LF is normalized at the tip with a value that is 6 times the value of the AGB (i) False positives. In the presence of random noise in the population (i.e., 6N stars) at its starting point one bin brighter output of our TRGB edge-detection response-function, there than the bin marking the discontinuity. The bin between the comes a point at which (a) the ﬂuctuations in the number of two terminal points deﬁnes the tip, and its value is the average detected stars (from bin to bin) and/or (b) Poisson noise in the of the two adjacent LFs (i.e., 3N stars). photometry of the individual stars themselves will produce The upper panel of Figure 1 shows the input LF binned into (spurious) features in the tip-detection and/or response- 9 histogram-like segments with bin No. 5 centered on the function output. These noise-induced features, if large enough, position of the discontinuity, marking the luminosity (in can be both qualitatively and quantitatively indistinguishable 3 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. from the expected signal (i.e., being positive deﬂections in the progressively increasing the photometric errors (at the tip) response-function, which have a similar width and relative from ±0.02, to 0.05, 0.10, 0.15, and 0.20 mag, respectively height when compared to the expected and/or true signal).In (corresponding to signal-to-noise ratios of 50, 20, 10, 6, and 5). the controlled simulations, discussed below, we quantify when In Figures 3–5, we then rerun parallel simulations, and where this situation starts to become a serious problem. progressively dropping the total population of stars by about (ii) Nondetections. Again, in the presence of excessive a factor of 10 each time: starting with about 120,000 RGB stars photometric noise, in particular, for small population sizes (or in the 1 mag interval below the tip (in Figure 2), and in a combination of the two), it is possible for the true signal to ending with a simulation having only 127 RGB stars in that become so weak that it is not detected at any signiﬁcant same 1 mag interval (in Figure 5). thresholding level, with respect to the ambient response-function In each of the next sections (each also containing four ﬁgures noise. This situation is fatal; but the circumstances under which and six main subpanels), we explore the effects of changing the it is likely to occur can be anticipated and identiﬁed using these smoothing (going up from 0.01 to 0.05, and ﬁnally 0.10 mag) simulations as a guide (see, for example, Figures 4 and 7). in Figures 6 through 9,at ﬁxed population sizes per ﬁgure and (iii) Potential bias in the tip detection algorithm. Given the increasing photometric errors through each of the subpanels, as unequal count rates of stars contributing to the LFs above and in the previous section. below the TRGB, it might be thought that even a symmetric We then close out in Section (4.1) holding the smoothing at a response-function kernel might return an asymmetric (i.e., ﬁxed value (at 0.10 mag) and assessing the effects of changing biased) answer, given that more RGB stars are moving across the population size in Figures 10 through 13, while changing the TRGB discontinuity to intrinsically brighter magnitudes the photometric errors in the subpanels within those ﬁgures. than there are bright AGB stars moving in the opposite This extensive grid of plots is provided both for their use as direction (across the TRGB discontinuity) to fainter magni- predictors in planning future observations, and for their use as a tudes. We investigate this potential source of systematic error guide in understanding the LFs and edge-detector output once (detector bias) throughout the simulations studied below. they are acquired. To put this into perspective for the 12 galaxies observed by Freedman et al. (2019) in their determination of a TRGB-based value of the Hubble constant, 4.2. Precision they detected an average of 4000 RGB stars in the 1 mag We are endeavoring to (a) measure the tip magnitude, (b) interval below the TRGB (with anywhere from 1000 to 20,000 measure its statistical uncertainty (its precision), and (c) RGB stars in individual cases, depending on the distance provide any estimate of bias (its accuracy) inherent in the modulus of the host galaxy and how far into the halo any given methodology explored here. A number of factors contribute to exposure was taken). As for the typical photometric errors at the outcome. Some of these factors can be controlled in the tip, the exposures were scaled to the approximately known advance while setting up the experiment and/or observation, distances, and they all have uncertainties at the tip of and some of them can be ameliorated later in the data analysis about ±0.10 in F814W (I band). This would roughly stage. For instance, the source-count population, the amount of correspond to the middle right panels of Figures 6–8. crowding, and the signal-to-noise ratio in the photometry can To help navigate the various simulations, we provide a guide each be controlled with foreknowledge of the approximate to their ordered content in Table 1. surface brightness of the region being targeted, knowing the approximate distance, and adjusting the total exposure time (or 5.1. Low Degree of Smoothing size of the telescope), within allowable and practical limits. The type of kernel employed in measuring the ﬁrst derivative of the 5.1.1. A Range of Photometric Errors: 120,000 RGB Stars, Fixed LF at and around the tip, and the amount of smoothing of the Smoothing ±0.01 mag data chosen to be applied to the data, in advance of the kernel response-function application, can both be controlled to We start this detailed discussion with a high-deﬁnition optimize the output of the detector once the data have been simulation of the LF beginning 1 mag above, and ending 1 obtained. We consider each of these parameters in turn. mag below the TRGB, where the discontinuity is set to M = 0.00 mag across of the simulations in this paper. This would correspond to M = −4.05 mag, which closely matches the value 5. RGB + AGB Computer Simulations currently adopted by the Chicago-Carnegie Hubble Program In this series of simulations, we explore changing a number (CCHP; Freedman 2021).The ﬁrst magnitude interval, above of parameters while holding others ﬁxed. These include the the tip, is populated uniformly as a function of magnitude by photometric errors and overall population size (Section 4.1), AGB stars. For examples of published ﬂat AGB LFs above the different smoothing sizes (Section 4.2), and amount of tip, see Beaton et al. (2019), their Figure 4, Hoyt et al. (2018), crowding and/or blending (Section 6). In Section 5,we their Figure 6, and Nikolaev & Weinberg (2000), the inset illustrate how the width of the smoothing function does not histogram to their Figure 4, and their description of it being carry information on the uncertainty of the tip measurement. “The off-bar LF shows only a mild increase in the source counts Here, we explore the systematics of changing the photo- at the location of TRGB, but has the same, roughly constant metric errors at the tip (from one simulation to the next) while proﬁle at Ks brighter than 12 mag, due to the AGB population, holding the population size and smoothing ﬁxed. visible in the other two luminosity functions.” At the TRGB With Figure 2, we start at one extreme: a very densely discontinuity, the RGB population turns on at an initial rate (of populated LF (about 120,000 stars in total) having minimal stars per magnitude bin) 6 times greater than the AGB density (0.01 mag) smoothing and very high-precision photometry, as above the tip (see Scolnic et al. 2023, where they calculate a shown in the ﬁrst (upper left) panel. We then work (left to right variant of this contrast ratio R, using bins 0.5 mag wide, above and top to bottom) through the observed effects of and below the tip, for a large number of GHOSTS galaxies, 4 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 2. Six subpanels illustrating the effect of increasing the photometric noise, (from 0.00 to ±0.20 mag) at ﬁxed smoothing (±0.01 mag) and extremely large populations of RGB stars (about 120,000). The lower portions of each of the six subpanels show the ﬁrst-derivative edge-detector output in both its uncorrected (thin black lines) and its noise-weighted (thicker black line) forms. See text for a detailed discussion of the trends. ﬁnding that it ranges from R = 4 to 7 as seen by the annotations is a GLOESS ﬁt with a Gaussian smoothing window of in their Figure 5). Thereafter, the binned number density of RGB 0.01 mag, making it a close approximation, at this ﬁne binning stars increases with a logarithmic slope of +0.3 (Menendez and/or smoothing, to a spline ﬁt through the individual data et al. 2002; Makarov et al. 2006). points. The vertical line at M = 0.0 mag marks the exact The upper left panel in Figure 2 shows our highest-ﬁdelity, position of the TRGB that is equidistantly ﬂanked, in the lower and most optimistic realization, consisting of 120,000 RGB panel, by two dashed lines (barely visible in this panel) that stars and some 20,000 AGB stars. The bin size is 0.01 mag, are ±0.01 mag apart, showing the highest attainable resolution giving a typical RGB population of 1200 stars per bin, leading of the data and the response-function. to an expected 2σ scatter of ±70 stars per bin (or ±6% 1σ,as Below the LF, in the lower part of the panel, is the ﬁrst- can be seen in the plot). The solid line passing through the data derivative response-function as applied to the discretely 5 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 3. Six subpanels illustrating the effect of increasing the photometric noise, (from 0.00 to ±0.20 mag) at ﬁxed smoothing (±0.01 mag) and moderately large populations of RGB stars (11,000). The lower portions of each of the six subpanels show the ﬁrst-derivative edge-detector output in both its uncorrected (thin black lines) and its noise-weighted (thicker black line) forms. See text for a detailed discussion of the trends. sampled and (minimally) smoothed luminosity data above it. agree at their respective (close-to-peak) values at the center of We use the Madore-Freedman5 (hereafter MF5) edge-detector the plot where the true and/or input value resides. As is evident described in Appendix A, which samples the LF at 11 from a casual inspection of the various plots, noise-suppression optimally weighted points symmetrically placed around the results in much reduced ﬂuctuations everywhere across the output bin. Two versions of the output function are shown: the magnitude range probed by the tip detectors, without any thin solid line is the raw response-function (RRF) of the MF5 obvious degradation (or improvement) of the sought-after ﬁlter, while the thick black line is the (inversely) noise- signal at the TRGB discontinuity. We do point out, however, weighted response-function (NWRF), as described in that the width of the untreated TRGB detection is both Appendix A. The two response-functions have been scaled to asymmetric and wider than the noise-suppressed response, 6 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 4. Six subpanels illustrating the effect of increasing the photometric noise, (from 0.00 to ±0.20 mag) at ﬁxed smoothing (±0.01 mag) and small populations of RGB stars (1200). The lower portions of each of the six subpanels show the ﬁrst-derivative edge-detector output in both its uncorrected (thin black lines) and its noise- weighted (thicker black line) forms. See text for a detailed discussion of the trends. where the latter has the expected width of ±0.01 mag, which in no measurable bias in the ﬁrst-derivative response-function turn is the sampling limit of the data. The solid line marks the being used to detect the TRGB. exact position of the TRGB, and the two ﬂanking solid lines are We do, however, want to emphasize that there is no pressing again separated by ±0.01 mag for visual reference. need for smoothing the data when in this high-population, The GLOESS ﬁt to the LF faithfully tracks the discontinuity high-precision-photometry portion of parameter space; the input at 0.0 mag, in the upper panel, and the response-function, noise-weighting is sufﬁcient in suppressing spurious signals, in the lower panel, peaks precisely at the midpoint of the while simultaneously sharpening the edge-detector response. M = 0.0 mag discontinuity, in all cases. At the resolution of the In the second panel of this same ﬁgure (top right), we begin data and the detector output (0.01 mag in both cases), there is to explore the effects of adding photometric errors to the 7 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 5. Six subpanels illustrating the effect of increasing the photometric noise, (from 0.00 to ±0.20 mag) at ﬁxed smoothing (±0.01 mag) and impoverished populations of RGB stars (124). The lower portions of each of the six subpanels show the ﬁrst-derivative edge-detector output in both its uncorrected (thin black lines) and its noise-weighted (thicker black line) forms. See text for a detailed discussion of the trends. individually observed stars contributing to the simulated LF. The only effect obvious to the eye is the rounding of the All of the other parameters (in this instance, population size, originally sharp shoulders of the LF immediately above and smoothing, and the detection kernel) used in the 12 subpanels below the magnitude of the TRGB discontinuity. The dashed of Figure 2 are kept unchanged. vertical lines in the upper panel mark the 1σ smoothing radius In this second simulation, randomly generated photometric inﬂicted on the discontinuity by the degradation of the errors, having a Gaussian σ of ±0.02 mag and a mean of zero, photometry. In the subpanel below the LF, we again show have been applied randomly to each of the sampled stars, which the MF5 response-function, in both the raw (thin solid line) and were then rebinned at 0.01 mag intervals, replotted, and the noise-suppressed (solid black line) forms. Again the RRF is reanalyzed. considerably noisier overall, and it is noticeably wider (with, 8 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 6. Six subpanels illustrating the effect of increasing the photometric noise, (from 0.00 to 0.20 mag) at ﬁxed, but slightly larger smoothing (0.05 mag) than previously discussed and again for very large populations of 120,000 RGB stars. The lower portions of each of the six subpanels show the ﬁrst-derivative edge- detector output in both its uncorrected (thin black lines) and its noise-weighted (thicker black line) forms. See text for a detailed discussion of the trends. noise-induced, broad wings) at the discontinuity. The noise- error of ±0.20 mag. What is progressively different is the corrected response-function still has the bandwidth-limited decreasing signal-to-noise ratio of both response-functions as natural width of ±0.01 mag. compared to the baseline noise, at the ﬁxed baseline width of The same general trends continue as we increase the the discontinuity-sampling kernel (MF5 in this case). As the photometric errors (from ±0.05 to ±0.20 mag) in the observed slope of the LF at the TRGB discontinuity softens remaining four (lower) panels; that is, the raw response is with increased photometric errors, the power in the ﬁrst always broader than the noise-suppressed response width, derivative across a ﬁxed magnitude interval drops, while the which is stable and effectively unresolved at the ±0.01 mag Poisson population-sampling noise in the baseline LFs, on level right up to and including the largest tested photometric either side of the tip, remains largely unchanged. We 9 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 7. Six subpanels illustrating the effect of increasing the photometric noise, (from 0.00 to 0.20 mag) at ﬁxed, but slightly larger smoothing (0.05 mag) than previously discussed and for moderately large populations of RGB stars (11,331). The lower portions of each of the six subpanels show the ﬁrst-derivative edge- detector output in both its uncorrected (thin black lines) and its noise-weighted (thicker black line) forms. See text for a detailed discussion of the trends. emphasize here that the lower, response-function plots have function toward fainter magnitudes may also be a generic been sequentially rescaled for clarity, roughly normalized by feature of added noise affecting the wings. The NWRF is the peak of TRGB response. unresolved in all of the instances, regardless of the input Summary 1. For very large populations of stars deﬁning the photometric errors. As the power in the response-functions fall LF around the TRGB, the RRF, and the NWRF, each is found (with increasing photometric errors), the noise on either side to be an unbiased indicator of the position of the discontinuity and surrounding the discontinuity begin to encroach upon and in the LF marking the position of the TRGB. The RRF is found become competitive in amplitude with the declining response at to slowly but systematically increase in width with increasing the true position. This degradation is noticeable at a photometric errors. A slight skewing of the RRF distribution photometric error of ±0.10 mag, and becomes problematic 10 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 8. Six subpanels illustrating the effect of increasing the photometric noise, (from 0.00 to 0.20 mag) at ﬁxed, but slightly larger smoothing (0.05 mag) than previously discussed and for small populations of RGB stars (1240). The lower portions of each of the six subpanels show the ﬁrst-derivative edge-detector output in both its uncorrected (thin black lines) and its noise-weighted (thicker black line) forms. See text for a detailed discussion of the trends. Table 1 Guide to Simulations: Figures 2–13 and Panels (a) through (f) RGB Stars Smoothing Error at TRGB Number 0.01 0.05 0.10 0.00 0.02 0.05 0.10 0.15 0.20 120,000 Figure 2 Figure 6 Figure 10 ab c d e f 11,331 Figure 3 Figure 7 Figure 11 ab c d e f 1240 Figure 4 Figure 8 Figure 12 ab c d e f 124 Figure 5 Figure 9 Figure 13 ab c d e f 11 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. thereafter, for higher values of the photometric errors. In all population starts to contribute to an upstream ambiguity. At cases, however, the noise-suppression is effective in damping this level of smoothing, population size, and photometric error, down this background noise by about a factor of 2 compared to the tip cannot be extracted from the noise. the raw response value (see the last three panels for the worst- Summary 2. For an RGB population of approximately case examples). At the two largest values of the photometric 10,000 stars, an unambiguous detection of the tip can be errors (±0.15 and ±0.20 mag in the bottom two panels), the assured with a photometric error of ±0.05 mag or less. At a noise-induced spikes in the response-function become sufﬁ- photometric error of ±0.10 mag, the ﬁrst-detected discontinuity ciently large with respect to the declining response at the is the true one with false positives rapidly developing at fainter known and/or true position, and false-positive detections start magnitudes, downstream. However, at a photometric error of to become a problem especially downstream of the true tip. ±0.15 mag and beyond, false positives overwhelm the signal in Noise-suppression helps to damp these ﬂuctuations down, but power and in number, both below and above the true tip. does not eliminate all of the false positives in the regime of large photometric errors (>0.15 mag). 5.1.3. A Range of Photometric Errors: 1000 RGB Stars, Fixed Smoothing ±0.01 mag 5.1.2. A Range of Photometric Errors: 11,000 RGB Stars, Fixed This simulation drops the RGB population to about 1000 Smoothing 0.01 mag stars, another factor of 10 below the previous investigation. Almost immediately, at a photometric error level of At this iteration, we drop the total population of stars ±0.02 mag, the power in the response-function at the tip has contributing to the LF by about a factor of 10 (down to 11,000 dropped to a level comparable to population noise in the RGB RGB and 2000 AGB stars), keeping the smoothing at a very LF. Several false positives are seen (in the middle left panel of low level (±0.01 mag) as above, while again assessing the Figure 3) downstream of the true TRGB. Noise spikes in the effects of increased photometric errors. RGB magnitude range are so frequent (at this smoothing) that It is important to note at this point that the effects of they can randomly appear around the tip without really being decreased population size and increased photometric errors are detections of the tip. Note the cluster of noise spikes well below causally independent of each other in the plotted LFs. At ﬁxed the known position of the TRGB in the lower left panel and precision in the photometry, downsizing the population size then again a spike somewhat brighter (and certainly stronger) can only decrease the number of stars in any given bin and than the tip in the adjacent, lower right panel. thereby increase the relative error ( NN ) in that bin. The Summary 3. For a population of only 1000 RGB stars, a increased scatter in all of the panels of Figure 3 as compared to photometric error in excess of ±0.02 mag results in false Figure 2 is a direct result of the decreased number statistics and positives overwhelming the tip detection, in the absence of any can be seen repeated and progressively ampliﬁed later on in signiﬁcant smoothing (but see Section4.2 below). Figures 4 and 5 as the population size decreases further. What may not be immediately obvious is why the photometric redistribution of the data across bins at a given 5.1.4. A Range of Photometric Errors: 124 RGB Stars, Fixed population size has virtually no affect on the noise amplitude in Smoothing ±0.01 mag the LFs, seen on either side of the discontinuity. The reason for As may well have been anticipated by the trends already this is that, while this form of smoothing redistributes data seen above in the increased number of false positives as the laterally, it does not signiﬁcantly change the local mean value sample size decreased and as the photometric errors increased of N in any given bin (i.e., photometric redistribution conserves (at ﬁxed smoothing), this last simulation (shown in Figure 5) total counts within its smoothing radius). That means, of contains only 120 RGB stars, and is dominated by noise. While course, that NN is also conserved. Photometric blurring of the six-to-one contrast ratio between the RGB and the AGB individual data bin does not reduce N population noise in the population still applies, the depleted populations on either side RGB continuum; however, because of the strong asymmetry, of the jump at the TRGB are so dominated by Poisson noise inherent in the jump in the LF at the TRGB, more RGB stars that (without smoothing) both the LF itself and the tip-detection migrate to higher luminosities (and boost the apparent AGB response-function are almost indistinguishable from noise. But population) than the other way around. Accordingly, photo- with hindsight, gleaned from the upcoming panels and ﬁgures, metric errors erode the tip and systematically decrease the slope there is still (surprisingly perhaps) meaningful information on of the transition marking the rise from the AGB to RGB the position of the TRGB in all of these realizations. populations, decreasing the contrast between the AGB and the Summary 4. RGB populations of this size are insufﬁcient to tip, but still not moving the position of the discontinuity. provide reliable measurements of the tip magnitude, but some The small degree of (±0.01 mag) smoothing in these information can still be gained. simulations tracks not only the population ﬂuctuations from bin to bin but also the precisely deﬁned, sharp rise marking the 5.2. Increased Smoothing TRGB. As the photometric errors increase and the transition widens and ﬂattens the population, the power in the response- 5.2.1. A Range of Photometric Errors: 120,000 RGB Stars, Fixed function crossing the everwidening transition region starts to Smoothing ±0.05 mag drop. From a photometric error of ±0.05 mag onward (middle left panel), it is approaching the noise level of the RGB We now repeat the cycle of exploring population size effects population noise. In this simulation, there are 3–4 noise spikes and photometric errors, but now at an increased level of downstream of the true tip that are of similar power, rendering smoothing of the data set to ±0.05 mag. the identiﬁcation of the true tip ambiguous. At a photometric Once again, returning to the upper left panel of Figure 6,we error of ±0.15 mag (lower left panel), the number density of begin with an RGB population of 120,000 stars below the tip false peaks is overwhelming, and even noise in the AGB and a photometric error of 0.00 mag. At this level of precision 12 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. in the data, the discontinuity occurs between two bins, and the 5.2.4. A Range of Photometric Errors: 120 RGB Stars, Fixed smoothing is inappropriately too large, needlessly degrading Smoothing ±0.05 mag the jump. Nevertheless, the power in the ﬁrst-derivative At our smallest population size of 120 RGB stars below the response-function (bottom section of the upper right panel) is tip, the advantages of smoothing are now becoming quite very high and well deﬁned, as one might expect. And its width apparent in the ﬁrst two panels of Figure 9, illustrating the is only ±0.01 mag. Increasing the error at the tip to ±0.02 mag onset of decreased photometric precision. The ﬁrst detected tip (upper right panel) widens the discontinuity somewhat, but the is the true peak, up to an error of ±0.02 mag, after which smoothing of ±0.05 mag is still too large. The output of spurious noise peaks overwhelm the detection both in advance response-function itself responds to the increased photometric of and beyond the true position of the TRGB. Conﬁdently errors by declining in power, and widening. In the middle left detecting the true position of the TRGB in RGB populations of panel, the smoothing and the photometric errors are identical, around 100 stars in the upper magnitude range can only be and the ﬁt at the tip is almost optimal. The response-function is done with high-precision data and is still risky, given that noise well centered, continues to widen with the increased errors, and spikes of comparable power are found systematically posi- can be seen to be starting to develop structured wings that are tioned up to ±0.1 mag above and below the true tip, in virtually due to increased, but smoothed, population noise downstream all of the realizations shown here. of the TRGB. In the ﬁnal (lower right) panel, the photometric Summary 7. Increased smoothing can help compensate for errors are at their maximum for this simulation (±0.20 mag), small population sizes if the photometric quality is very good. and the response-function is widened both by the smoothing of However, using small populations is still not advisable. the discontinuity in the LF plane and by the encroaching population noise, smoothed out in the response-function plane. 5.3. Largest Smoothing Considered It is noteworthy that throughout this simulation the mode of the response-function at the true TRGB luminosity is stable at the 5.3.1. A Range of Photometric Errors: 120,000 RGB Stars, Fixed 0.01 mag level despite the widening of the response output and Smoothing ±0.10 mag the asymmetric growth of its wings. Summary 5. A larger smoothing of 0.05 mag is too large for As we now move to overly aggressive smoothing, this large data with very small photometric errors. However, this smoothing population (120,000 RGB star) simulation is clearly being becomes more appropriate when the photometric errors are oversmoothed at the tip, up to the point that the smoothing and comparable to the smoothing value. With a large population of the photometric error at the tip are of equal magnitude, RGB stars, the tip location is extremely stable in all cases. ±0.10 mag in this case. As can be seen in Figure 10,the width of the response-function at high signal-to-noise is controlled by the adopted smoothing up to the cross-over point of smoothing 5.2.2. A Range of Photometric Errors: 11,000 RGB Stars, Fixed and photometric errors (middle right panel) after which the width Smoothing ±0.05 mag grows with the increased photometric errors (last two panels).At Despite the underﬁtting of the data around the TRGB (due to high values of the photometric errors, the earlier-mentioned the larger smoothing), the ﬁlter response is sharp, unambig- wings and low-level asymmetries are still present but obviously uous, and unbiased for photometric errors less than ±0.05 mag smoothed. Again, no bias is detected in the response-function. (which coincidentally corresponds to the adopted smoothing Smoothing offers little or no advantage in the detection or here) for an RGB population size about 11,000 stars. At high measurement of the tip discontinuity in this particular scenario. photometric errors (i.e., in excess of ±0.10 mag), false Summary 8. High levels of smoothing are not advantageous positives predominate upstream (middle right panel of Figure 7) for large populations of RGB stars. but eventually crowd around and compromise the integrity of the true tip detection, encroaching both from fainter and 5.3.2. A Range of Photometric Errors: 11,000 RGB Stars, Fixed brighter magnitudes. For instance, the strongest peak in the Smoothing ±0.10 mag lower left panels is due to a smoothed version of a clustering of random noise peaks two-tenths of a magnitude below the true As in the example discussed above, dropping the RGB TRGB. The existence of a peak at the correct position in the sample to 11,000 stars in Figure 11 does not quantitatively lower right panel cannot be given much credibility given the change the description of the response-function to increased ambient noise. photometric errors. However, there is now the ﬁrst indication Summary 6. The photometric errors greater than ±0.10 mag that the mode of the response-function output is being drawn cause the false positives and potential bias in the (blended) tip off center (at the ±0.05 mag level) by the increased noise in the magnitude for populations of 11,000 RGB stars. Increased smoothed wings (last four panels). Oversmoothing should be smoothing does not mitigate this effect. carefully monitored. Running through a range of smoothing parameters can alert the user to systematic errors being introduced because of oversmoothing noise into the true peak, 5.2.3. A Range of Photometric Errors: 1200 RGB Stars, Fixed as illustrated here in the last three panels. Smoothing ±0.05 mag When numerous (comparably signiﬁcant) peaks are found Dropping the sample size by another factor of 10, down to with a low degree of smoothing, no amount of additional around 1200 RGB stars below the TRGB, does not smoothing will reveal the true peak, but rather the resulting substantially change the description of the situation as given detection will be a weighted average of the surrounding peaks, in the previous section. The detection of the tip, as seen in in which may (with enough smoothing) appear to be a single Figure 8, is relatively strong and unambiguous up to a (broad) peak; it will probably be biased: consider smoothing the photometric error of ±0.05 mag after which competing false last three panels in Figure 7, as then seen in Figure 11.Our positives begin occurring above and below the true tip. recommendation is that future investigators always try a number 13 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 9. Six subpanels illustrating the effect of increasing the photometric noise, (from 0.00 to 0.20 mag) at ﬁxed, but slightly larger smoothing (0.05 mag) than previously discussed and for impoverished populations of RGB stars 124). The lower portions of each of the six subpanels show the ﬁrst-derivative edge-detector output in both its uncorrected (thin black lines) and its noise-weighted (thicker black line) forms. See text for a detailed discussion of the trends. of smoothing kernels bracketing their preferred solution so as to Oversmoothing in this context tends to occur when the reveal the presence (or absence) of substructure that a high smoothing parameter is in excess of the photometric errors at degree of smoothing would otherwise gloss over. the TRGB. In addition, we note that a wide range of edge- Real-world investigations into selecting an optimal smooth- detection methods using different smoothing kernels (and even ing have been undertaken by Beaton et al. (2019); see their including those using maximum-likelihood ﬁtting techniques) Figures 5 and 8 for examples of the implementation of an was found to agree to very high (0.01 mag) precision when iterative smoothing analysis. There, one can see solutions that applied to the TRGB data for IC 1613 (Hatt et al. 2017). The are oversmoothed systematically drifting from their less- two papers both offer a quantitative means of selecting an smoothed solutions, being drawn away by adjacent, individu- optimal smoothing parameter, which is the one that minimizes ally low signiﬁcance, but sometimes numerous peaks. the quadrature sum of the random and systematic errors, 14 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 10. Six subpanels illustrating the effect of increasing the photometric noise, (from 0.00 to ±0.20 mag) at ﬁxed, but moderate smoothing (±0.10 mag) and for very large populations of RGB stars (120,000). The lower portions of each of the six subpanels show the ﬁrst-derivative edge-detector output in both its uncorrected (thin black lines) and its noise-weighted (thicker black line) forms. See text for a detailed discussion of the trends. generally selecting smoothing parameters that are indeed close 5.3.3. A Range of Photometric Errors: 1200 RGB Stars, Fixed Smoothing ±0.10 mag to the measured photometric errors reported for stars at the tip. Summary 9. Oversmoothing can introduce a systematic bias At 1200 stars in the RGB, tip detection is unbiased and in the presence of noise, and should be cautiously examined. unambiguous at high signal-to-noise in the photometry at the tip (upper left panel of Figure 12). At lower photometric precision, adjacent noise spikes broaden and can bias the true It should be made clear that the IC 1613 data set and its reduction are exquisite in nature given the very high precision of the photometry and the tip detection by up to 0.1 mag (bottom two panels). In this sharpness of its tip. If similar investigations were to be shown for galaxies with realization, several of the deﬂections are toward brighter lower-quality data (say due to their increased distance or mixed populations), magnitudes, but there is no reason to believe that these are they would be unlikely to demonstrate such a high level of agreement. 15 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 11. Six subpanels illustrating the effect of increasing the photometric noise, (from 0.00 to ±0.20 mag) at ﬁxed, but intermediate smoothing (±0.10 mag) of a moderately large population of RGB stars (11,000). The lower portions of each of the six subpanels show the ﬁrst-derivative edge-detector output in both its uncorrected (thin black lines) and its noise-weighted (thicker black line) forms. See text for a detailed discussion of the trends. anything more than random ﬂuctuations around the mean (see Figure 13), the true peak is properly detected, but it is not below). the highest peak over the 2 mag interval. In this particular simulation, the strongest (all false positive) peaks are found, four out of six times, at fainter magnitudes than the true tip, 5.3.4. A Range of Photometric Errors: 120 RGB Stars, Fixed and by up to 0.35 mag separation. With an average of one Smoothing ±0.10 mag star per RGB bin, wild statistical ﬂuctuations, both in the LF itself and in the discontinuity detector, are both to be This ﬁnal realization shows the ﬁlter response to a small sample size (120 RGB stars) with large (±0.10 mag) expected and are seen. This is far from being an acceptable smoothing applied to monotonically increasing photometric situation for detecting or measuring the TRGB with any errors. At high signal-to-noise (the top two panels of degree of conﬁdence. 16 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 12. Six subpanels illustrating the effect of increasing the photometric noise, (from 0.00 to ±0.20 mag) at ﬁxed, but intermediate smoothing (±0.10 mag) of a moderately large population of RGB stars (12,000). The lower portions of each of the six subpanels show the ﬁrst-derivative edge-detector output in both its uncorrected (thin black lines) and its noise-weighted (thicker black line) forms. See text for a detailed discussion of the trends. Summary 10. One should not even attempt a tip detection at function, and it alone carries little or no quantitative information low signal-to-noise in situations where the population size is on the uncertainty in the tip measurement itself. This only in the hundreds. Spurious signals will be found above and nonresponse of the width of the Sobel Function output to below the true tip. variations in smoothing and population size is shown in the three panels in Figure 14. At the bottom of each of the ﬁgures, 6. Exploring Smoothing versus Tip Uncertainty we show the Sobel-ﬁlter response to the smoothed LFs plotted above them. The two thin vertical lines centered on the In Hatt et al. (2017), and in Jang et al. (2018), it has been response-function are not determined by the the Sobel-ﬁlter shown that the output width of the Sobel response-function (and response itself, but rather they mark the input width many of its variants) is dominated by the width of the smoothing 17 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 13. Six subpanels illustrating the effect of increasing the photometric noise, (from 0.00 to ±0.20 mag) at ﬁxed, but intermediate smoothing (±0.10 mag) and an impoverished population of RGB stars (120). The lower portions of each of the six subpanels show the ﬁrst-derivative edge-detector output in both its uncorrected (thin black lines) and its noise-weighted (thicker black line) forms. See text for a detailed discussion of the trends. (±0.025 mag) of the GLOESS smoothing function, which has keeping the GLOESS smoothing the same as that in the middle nothing to do with any of the observed properties of the data. panel. The width of the response-function is unchanged, as These lines match the observed width of the Sobel-ﬁlter shown by the predicted width based on the GLOESS smoothing. response-function because the smoothing dominates. To see For a vastly different number of data points, there is no this, in the central panel, the GLOESS smoothing width has qualitative change in the width of the response-function. We end been doubled to ±0.050 mag, and the response-function is seen with where we started: the width of the Sobel-ﬁlter response- to have exactly doubled as well; same data, same population function has little or no discernible information content on the size, but twice the width of the response. The ﬁnal (right) panel uncertainty of the measured tip magnitude, and it should not be shows the effect of reducing the LF population by a factor of 10, used indiscriminately in any such applications. 18 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 14. The direct dependence of the width of the Sobel Function response to variations in smoothing (comparing the left and middle panels) and insensitivity to population size (comparing the middle and right panels). The left panel shows the Sobel-ﬁlter tip-detection response to the TRGB luminosity function, which is shown rising diagonally across the top of the plot. The two thin vertical lines mark the input width (±0.025 mag) of the GLOESS smoothing function. They match the observed width of the Sobel-ﬁlter response-function. In the central panel, the GLOESS smoothing width has been doubled to ±0.050 mag, and the response-function is seen to have doubled as well. The ﬁnal (right) panel shows the effect of reducing the luminosity function population by a factor of 10, keeping the GLOESS smoothing the same as the middle panel. The width of the response-function is unchanged, as shown by the predicted width based on the GLOESS smoothing. However, two other observables, population size and made for that study giving the uncertainty for that particular tip measurement. Failing that, these plots can be interpolated for a photometric errors, do have an expected and predictable impact ﬁrst-order estimate of the uncertainty to be associated with the on the uncertainty of the TRGB measured magnitude. This is choice of smoothing, measured population size, and known illustrated in the three panels in Figure 15 where the run of photometric errors (labeling each of the three realizations in (statistical) edge-detection errors with population size and with each of the three smoothing plots). photometric errors is shown. In each of the panels, the vertical There is one ﬁnal caveat. The simulations in this section axis gives our desired output: the standard deviation of the were undertaken for situations wherein crowding is not a major measured TRGB magnitudes. The plotted points are derived source of error at the tip. However, if observations are made in from 200 independent realizations of the LF, each smoothed regions of high surface brightness (and commensurately higher and individually scanned by our edge-detection ﬁlter. The three levels of crowding), then the simulations presented in the next line-linked symbols each track the increased precision in the section should take precedence. TRGB measurement as a function of increasing sample size for a given mean error in the TRGB photometry. The horizontal axis tracks the population size of the LF being scanned, deﬁned 7. Crowding Simulations here by the number of RGB stars in the 1 mag interval Our ﬁnal set of simulations targets the important question immediately below the TRGB. The ﬁlled circles have concerning the effect of source crowding on the RGB LF, photometric errors of ±0.10 mag. The circled plus signs especially at and around the discontinuity deﬁning the tip. have ±0.05 mag errors. And the circled dots represent error- In order to isolate and unambiguously determine the effects free photometry. The amount of GLOESS smoothing increases of crowding on the LF above the TRGB, we have set the AGB by factors of 2 (across the three panels), ranging from ±0.05 population to zero, which would ordinarily be found in the 1 to ±0.20 mag top to bottom. mag interval above the tip. We did this intentionally so as to Modulo the Poisson noise inherent in these ﬁnite simula- show what the crowded population looks like unhindered by tions, the trends are clear: All of the determinations of the superimposing it on a true AGB population; i.e., to see the uncertainty in the TRGB measured magnitude decrease signature of crowding alone. In each of Figures 16–20,we monotonically with increased sample size. And all trend lines show (in the left-hand panel) the input LF as a blue-shadowed decrease more slowly as the photometric errors increase. white line rising in number abruptly at M = −4.00 mag and Moving from ﬁgure to ﬁgure, the effects of the GLOESS thereafter exponentially increasing to form the RGB LF. The smoothing are seen to be present but subdominant. In any case, actual numbers of stars deﬁning the simulated LF are shown by for any given observation, the three parameters controlling the the black-shadowed red line, sampled at 0.01 mag intervals. In calculated uncertainty on the TRGB magnitude (the population the right panel, we show the output of the edge-detector in size, the photometric errors, and the adopted smoothing) are yellow. Running between the panels is a thin black (horizontal) known and can be input into a numerical simulation tailor- line marking the input TRGB discontinuity. In the right panel, 19 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 15. Variations of the run of edge-detection errors with population size and with photometric errors, as a function of the GLOESS smoothing changing from here through Figure 17. The vertical axis gives the standard deviation of the measured TRGB magnitudes derived from 200 independent realizations of the luminosity function for each plotted point. The three line-linked symbols each track the decrease in the TRGB uncertainty as a function of increased sample size for a given mean error in the TRGB photometry. The horizontal axis is gives the population size normalized by the number of RGB stars in the 1 mag interval immediately below the TRGB. Filled circles have photometric errors of ±0.10 mag. Circled plus signs have ±0.05 mag errors. And circled dots represent error-free photometry. The amount of GLOESS smoothing increases by factors of 2 from this to the next two ﬁgures. Here, we show the results for a ±0.05, ±0.10, and ±0.20 mag smoothing. Ripples in these trends are not signiﬁcant but due to small number statistics in the individual simulations. the position of the discontinuity as measured by the edge- 8. Eliminating AGB Stars detector is shown as a dashed black line. Finally, in the left- hand panels, in the otherwise empty space above the TRGB, we Although AGB stars above the tip are naturally found in most have inset the simulated CCD image used in the calculation halo ﬁelds, they have a fairly ﬂat LF that only serves to decrease (see the captions for the key to the various symbols marking the contrast of the tip discontinuity by placing the RGB LF on a stars in the image). slightly elevated background; but that loss of contrast does not The aforementioned simulated CCD image was instru- impose a bias, only a decrease in precision. If one considers the mental in undertaking the crowding simulation. The image TRGB discontinuity as a step function, then it is easy to visualize consists of a square array of 1000 by 1000 elements. As stars that, for reasonable high levels of the contrast ratio R > 3, say, the populated the (smooth blue line) inputLFseeninthe left measurement of the onset (when approached from brighter panel, they were randomly assigned a cell in the image, their luminosities) is not inﬂuenced by level of the baseline and/or magnitudes were converted to ﬂuxes, and they were added to background upon which it is being measured. The AGB that cell. If the cell was already occupied, the ﬂux was contribution, close to the tip, can be thought of as a relatively augmented, and the cell was considered to be crowded. The constant pedestal upon which the TRGB discontinuity is detected. process was continued until all stars were assigned to cells. The presence of true AGB stars can be eliminated by time- The summed ﬂuxes were then converted back to magnitudes, domain observations of the TRGB ﬁelds. The Gaia Mission has and these magnitudes were rebinned (at 0.01 mag resolution) shown that virtually all true AGB stars in this luminosity range thereby creating the crowded LF shownbythe jaggedred are variable (see Figure 3 in Eyer et al. 2019 for the types of line in the LF plot. variables in the Gaia CMD; and especially our Figure 8, which Summary 11. It is apparent from these simulations that self- gives the fraction of variables sitting at unity, red points seen crowding blurs the tip discontinuity. In addition, high levels of directly above the downward slanting TRGB; while no RGB crowding can cause a bias in the measured tip magnitude. In stars, at or below the tip, have been found to show variability general, it is preferred to make these measurements in low- density halo ﬁelds to avoid crowding issues. greater than 0.04 mag full amplitude; black points in the 20 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 16. A simulation of the RGB luminosity function consisting of 117,000 stars populating the magnitude range from M = −2.0, to a sharp cutoff at M = −4.0 mag deﬁning the TRGB. The individual stars are assumed to be error-free in their individual magnitudes. Noise in the luminosity function is entirely due to Poisson noise in the counts in the individual bins. Output from the Sobel-Pascal-7 edge-detector is shown in the right panel. The black horizontal line, spanning the two panels, is the TRGB magnitude set to M = −4.00 mag. The dashed black horizontal line in the right panel only is the mean value of the response-function. The inset “Simulated CCD Image” in the left panel shows the positions of all stars used in the simulation. Large yellow dots, circled in red and black, are all RGB stars from M = −4.00 to −3.90 mag. Crowded stars brighter than the TRGB are shown as larger black ﬁlled circles. This plot is given to provide a visual impression of the self-crowding of stars near the TRGB that results in the small population of stars above the tip in this particular luminosity function. aforementioned Figure 8); they would then at most contribute a side, and increase the contrast of the discontinuity deﬁning the TRGB as approached from either side. ±0.01 mag blurring of the tip. Identifying and removing the variable AGB population will aid in deblurring and decontaminating the tip from the bright 9. Unaddressed Issues: Star Formation History of the Halo on the Position and Color Dependence of the TRGB The referee has correctly pointed out that Soszynski et al. (2004) have a paper entitled “Small Amplitude Variable Red Giants in the Magellanic In a paper by Brown et al. (2003), the case has been made for Clouds.” In that paper the largest amplitude variables are above the TRGB and a signiﬁcant population of intermediate-aged (6–8 Gyr), high- consist of AGB stars alone (long-period variables, Miras, and semiregular metallicity ([Fe/H] > 0.5) RGB stars being present in the halo variables); below the tip, the amplitudes systematically decrease with period (see examples in their Figure 4), and the authors believe that these fainter of Messier 31 (M31). These stars can exceed the luminosity of variables are a mix of AGB and RGB stars. They name these stars Optical the old, metal-poor (standard) TRGB population, but they also Gravitational Lensing Experiment Small Amplitude Red Giants (OSARGs).In ascend at a color that is far to the red of the most metal-rich, old their Figure 2, the TRGB can be found at W ∼ 11.5 mag, and at TRGB stars (see their Figure 1(f)) The application of both a 1.5 < (V − I) < 2.4 in their Figure 3. In their Figure 2, cutting the lower left panel in color restricts the RGB subtip population to stars that have log blue and (especially in this case) a red cutoff to the RGB stars, P < − 1.8. Applying that cut to the period–amplitude plot in the panel directly being used to detect the tip, effectively deals with these stars. In above the period–color plot reveals that the OSARGs below the tip have peak- any case, in a forthcoming paper (Freedman et al. 2023),itis to-peak amplitudes starting at 0.04 and dropping to 0.01 mag. Converting amplitudes to equivalent σ then suggests that these very-small-amplitude shown that a comparison of TRGB distances with Cepheid variables contribute no more scatter than ±0.010–0.003 mag. The claim in the distances to the same galaxies gives a combined scatter of only subsequent literature (Anderson et al. 2023) that “every star at the TRGB is ±0.066 mag, which must bracket the total impact of all variable” is true, at the millimag level, but it is not of concern in the context of the TRGB extragalactic distance scale. remaining random errors, including the scatter that might be 21 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 17. Same as Figure 16 except this simulation involve slightly over 200,000 RGB stars in the frame. Crowding is now quite apparent in the inset CCD image, and has resulted in an appreciable population of crowded stars falling in number from the tip to 0.75 mag brighter where the self-crowding of two stars at the tip would appear as one unresolved source. For clarity, however, only one star in 10 is plotted. See text for additional details. The offset between the blue (input) line and the (red) output line in this and in the following three ﬁgures is a direct consequence of the crowding systematically producing more stars at a given magnitude by merging two or more fainter stars. introduced by star formation history difference between these than R = 15 kpc to each galaxy, and especially along the major halos. axis, which are dominated by disk stars.” Additionally, a concern about the presence of young Theory also suggests that a younger population would have populations of red giant stars in the halos of galaxies has been little impact even if it were mixed in. From model predictions of raised in the literature, most recently and extensively by McQuinn et al. (2019), it is expected that the F814W absolute McQuinn et al. (2019). Their message is mostly cautionary. We luminosity of the TRGB should have a small dependence on agree with that stance, but offer up a number of lines of stellar age of roughly 0.02 mag across an age range of 5 Gyr and evidence arguing for optimism that the effect of younger 0.04 mag across 0.5 dex. Indeed, inspection of the lower panel in populations (star formation history of the halo), if present, is a Figure 2 of McQuinn et al. shows that the effect of age spread is minor contributor to the measured scatter in the TRGB, indeed small, but it is also degenerate in its correlation with metallicity. Furthermore, this point has also been recently especially in the I band. Some of the concern about young populations interfering addressed in the single-authored paper by Hoyt (2023) where he with the detection and measurement of the TRGB is driven by states, “a long-standing question of the TRGB concerns the extent ill-fated applications of the TRGB method too close to the disk. to which age can shift the observed colors and magnitudes of The following quote from the GHOSTS Team (Monachesi TRGB stars, potentially breaking the assumption of universality in et al. 2013) summarizes the situation very well: “The CMDs any single proposed calibration (Salaris & Cassisi 2005).” are mostly populated by old RGB stars (older than 1 Gyr). Encouragingly, in this section, it was shown that the Jang & There are however younger populations such as blue, extended Lee (2017) quadratic-tip color dependence—based on observa- main-sequence (MS) stars (<500 Myr) or massive stars tions in the stellar halos of L galaxies—describes very well the burning helium in their core (25–600 Myr old red and blue TRGB magnitude–color relation in the modulation collimators loop sequence stars). These appear primarily in the ﬁelds closer (this study), Local Group dwarfs, and M33 (Rizzietal. 2007). 22 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 18. Same as Figure 16 except this simulation involves slightly over 700,000 RGB stars in the frame. Crowding is now quite apparent in the inset CCD image, and has resulted in an appreciable population of crowded stars falling in number from the tip to 0.75 mag brighter where the self-crowding of two stars at the tip would appear as one unresolved source. See text for additional details. This consistency indicates that for RGB stars found in these the cumulative holdings of TRGB distances at NASA Extra- 11 12 environments either the age distributions are identical or the age- galactic Database and Extragalactic Distance Database ). dependent variations in the I-Band TRGB magnitude are minimal. We stress that, for accuracy and precision, the method should In either case, the I-Band TRGB appears well-behaved and be applied in the outer halos of galaxies, where the effects of without a measurable bias across these host environments. The extinction and self-crowding of TRGB stars are minimal. With bottom line is as follows: if you detect blue MS stars or red these simulations, we have shown the trade-offs between a supergiant populations in your ﬁelds, then you are clearly too number of factors, including photometric precision, numbers of close to the disk, and any attempt to determine a TRGB stars deﬁning the RGB LF, and the effects of crowding. These distance to any such a line of sight through the galaxy is subject simulations can be used as a guide to optimize the choice of to systematic effects. Stay as far as possible out into the pure halo ﬁelds for accurate TRGB measurements. halo, preferably along an extension of the galaxyʼs minor axis. The above simulations presuppose that observations of the TRGB for the purpose of extragalactic distance determinations 10. Summary, Conclusions, and General Advice are being made in the halos of galaxies. Thus, they are well away from disk contamination consisting of dust, gas, and stars TRGB distances have become one of the most precise and accurate means of measuring the distances to galaxies in the of mixed ages, colors, and spatial densities. This contamination nearby universe (see, for instance, Dalcanton et al. 2009; can only degrade the TRGB detection and act (in the case of Karachentev et al. Freedman et al. 2019; Anand et al. 2022;and dust extinction) in biasing the apparent magnitude of the tip to fainter magnitudes. In terms of the lack of bias due to the AGB stars observed in these In a review of TRGB modeling, McQuinn et al. (2019) state, simulations, we note that Hatt et al. (2017) had already remarked on this “Given the building histories of halos, it is reasonable to expect (noneffect) in their own independent simulations, stating, “K we ﬁnd that the variations in ages and metallicities.” They then go on to say, AGB component simulated here has no substantial effect on the measured TRGB magnitude. The ratio of TRGB to AGB stars near the tip is about 4:1, “Assuming stellar halos are consistently metal-poor with little which might, conceivably, cause a TRGB measurement to be systematically brighter. Nonetheless, we ﬁnd that the signal-to-noise of the TRGB still https://ned.ipac.caltech.edu outweighs the noise component due to AGB stars[,] and there are minimal systematic effects.” https://edd.ifa.hawaii.edu/ 23 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 19. Same as Figure 16 except this simulation involves slightly over 1,370,000 RGB stars in the frame. Crowding is now quite apparent in the inset CCD image, and has resulted in an appreciable population of crowded stars falling in number from the tip to 0.75 mag brighter where the self-crowding of two stars at the tip would appear as one unresolved source. For clarity, only one star in 10 is plotted. See text for additional details. variation does not appear to be valid”; and they add, “To date, largely degenerate with the color dependence of the TRGB few constraints have been placed on the stellar ages.” However, luminosity on metallicity. The situation may be less straight- in their conclusions, they more optimistically state, “In the I- forward at longer wavelengths. JWST observations will be band equivalent Hubble Space Telescope F814W ﬁlter and extremely important here. JWST F090W ﬁlter, the TRGB is remarkably constant across But what do the observations of the TRGB have to say on all ages and metallicities probed.” We feel that it would be this matter? Figure 5 of Freedman et al. (2019) gives a remarkable that any halo would not have a ﬁrst generation of comparison of TRGB distances with Cepheid distances to a low-metallicity red giant stars. These stars will be the brightest variety of galaxies of different star formation histories (ages), TRGB stars in the I band and will trigger the edge-detector different mean metallicities, different distances, and different before any other higher-metallicity (potentially fainter) popula- amounts of reddening. For the entire ensemble (near and far), the combined scatter is only ±0.11 mag, which, if equally tion would enter the mix. That is to say, if there is a population of fainter, high-metallicity stars in any given halo, along with shared between the two methods, would imply that they are the generations of lower-metallicity stars that gave rise to them, each good to 4% in distance. However, if you just look at the in the marginalization process undertaken before measuring the closest sample, their σ drops to ±0.05 mag, which means that tip, the high-metallicity stars will be systematically below the the two methods are each good to 2% in distance. The ﬁrst triggering of the edge-detector and will simply augment takeaway message is that inside of that 2% all of the unresolved the RGB LF without a signature of their own (see Figure 12 in or unknown systematics are themselves contained at that same Hoyt 2023). Similar arguments can be made for the color- level, be it metallicity, age, or biased ﬁtting methods. On that rectiﬁed TRGB at longer wavelengths if the curvature down- note, we are optimistic. Having population sizes that are sufﬁcient to ﬁll the RGB ward to fainter magnitudes persists at higher metallicities (which is strongly correlated with color). Finally, it needs to be luminosity up to and including the tip is crucial to the emphasized that, as Figures 2 and 3 in McQuinn et al. (2019) extraction of an unbiased TRGB magnitude. For example, for vividly demonstrate, the color and luminosity dependence of RGB populations of less than 1000 RGB stars in the ﬁrst the TRGB on age is extremely small (in the I band), and it is magnitude interval below the true TRGB, false detections of 24 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 20. Same as Figure 16 except this simulation involves slightly over 2,000,000 RGB stars in the frame. Crowding is now quite apparent in the inset CCD image, and has resulted in an appreciable population of crowded stars falling in number from the tip to 0.75 mag brighter where the self-crowding of two stars at the tip would appear as one unresolved source. See text for additional details. the tip can be expected at the ±0.1 mag level when the of the TRGB discontinuity, and eventually bias the tip photometric precision of the data is worse than ±0.05 mag (see detection (to bright magnitudes). However, as the simulations the lower six subpanels in Figures 8, 9, 12, and 13). in Figures 16 through 20 clearly demonstrate, this effect can be Degradation of the tip due to increased photometric errors predicted by the source density of RGB stars in any given ﬁeld. can be compensated for by having increased population sizes The attempts to increase population statistics of the RGB by (compare Figures 2 and 6 with 3 and 7). moving into higher surface brightness regions of the inner halo We have demonstrated that it is best to use the least amount should be tempered because of this self-crowding effect, in of smoothing possible, commensurate with the photometric addition to line-of-sight extinction issues within the disk (that errors and population sizes. When numerous (comparably are not included in this simulation). signiﬁcant) peaks are found with a low degree of smoothing, no In the end, the characteristic (exponentially increasing) LF of amount of additional smoothing will reveal the true peak, but the faux AGB stars will betray their presence, and signal rather the resulting detection will be a weighted average of the impending bias. This could, in principle, be modeled away, but surrounding peaks, which may (with enough smoothing) might best be avoided by not observing in high-surface- appear to a be a single (broad) peak; it will probably be brightness regions to begin with. However, we do caution biased: consider smoothing the last three panels in Figure 6,as against smoothing data that are in the self-crowding regime. then seen in Figure 10. Our recommendation is that future Smoothing Figures 19 or 20 would only lead to (unnecessarily) investigators always try a number of smoothing kernels biasing the edge response to brighter magnitudes. Irreparable bracketing their preferred solution so as to reveal the presence damage to the tip detection is seen in the highest degree of self- (or absence) of substructure that a high degree of smoothing crowding simulated in Figure 20. It too could be modeled; but, would otherwise gloss over. See Figures 5 and 8 of Beaton the best solution would be to reobserve the galaxy in a region et al. (2019) for a recent implementation of this iterative of signiﬁcantly lower surface density of stars. smoothing analysis. We have simulated the CCHP adopted smoothing and Similarly, self-crowding of RGB stars near the tip results in a ﬁltering of the AGB and/or RGB LF that is being used to population of false AGB stars, which also decrease the contrast measure the magnitude of the TRGB. We ﬁnd that the width of 25 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. the Sobel-ﬁlter response-function is totally dominated by the Smoothing can take various forms. They can range from preceding GLOESS smoothing of the LF. We have, however, simple, moving rectangular averages (equally weighted) over a calibrated the run of the uncertainty in the measured value of ﬁnite number of adjacent pixels, to a more sophisticated the TRGB as a function of (a) population size and (b) smoothing using triangular, biweight and/or higher-order uncertainties in the stellar photometry at the tip. Epanechnikov weightings, all of which symmetrically decline For readers wishing to get a sense of the uncertainties on their over a ﬁnite range of pixel support and are identically zero distances in advance of making the observations, we suggest everywhere outside that range (see, for example, Silver- consulting the three panels in Figure 15. They will also be useful man 1986; also Jahne 1991). By the central limit theorem, in validating the observed error on the tip after the population multiple applications of any of these smoothing kernels size and error at the tip have been empirically deﬁned. converge on a Gaussian. A discretely sampled (digital) Finally, it should be noted that all of the recommendations Gaussian is itself, of course, a smoothing kernel (but one of being derived from these simulations (implicitly for the I band, inﬁnite support, in principle). where the run of TRGB magnitude is ﬂat with color) apply One of the earliest applied and certainly one of the most equally well to those other wavelengths once the CMDs have elementary quantized edge-detection kernels is the so-called been rectiﬁed. By rotating the data using predetermined slopes Sobel ﬁlter. This ﬁlter involves the simple differencing of pixel of the TRGB, the respective tips will also show no trend of intensity values on either side of a target position. The Sobel magnitude with color. The rectiﬁed magnitudes can then be ﬁlter, in one-dimension, takes the normalized form of [−1, 0, marginalized, and an edge-detection can be applied to the +1]. Indeed, this is the ﬁrst kernel in Figure 21 (named MF3 resulting color-corrected LFs. In support of this, recent articles, and shown in Panel (A)). At the other extreme, the ﬁrst DoG is (purely observational and mixed with modeling), both Wu et al. also a highly effective gradient detector. Invoking the binomial (2014), their Figure 5, and Durbin et al. (2020), their Figure 3, theorem once again, we recall that for very small samples show that in the near-infrared F110W (J band) there is a clearly Pascal’s triangle gives the binomial terms’ integer numbers of linear trend of TRGB luminosities with the color at least over ﬁnite-support sampling (progressively approximating, and the bluest colors ranging from 0.70 < (F110W − F160W) < eventually converging upon a Gaussian). That is well known. 0.95 mag. What is not commonly stated, but must be equally true, is that There are many additional sources of statistical and the differences between adjacent binomial terms are then systematic errors that these simulations have not explicitly discretely sampled approximations to the ﬁrst DoG. included. These uncertainties could stem from issues in Evaluating the location of the tip can then be done in assumed point-spread function (PSF) libraries, charge-transfer either of two equivalent ways: (i) ﬁnd the value of x where efﬁciency corrections, etc. While improving with time, some the output of the Sobel ﬁlter is a maximum, or (ii) ﬁnd the fraction of these issues still persists. And on top of this, there value of x where the output of the slope of Sobel ﬁlter is ﬂat. are additional systematics when dealing with PSF photometry, The Sobel ﬁlter is the ﬁrst derivative of the input function; such as errors in aperture corrections, or mismatching PSFs the latter is the second derivative of the input function—it is (due to telescope focus shifts, breathing, etc.). The list goes on, commonly known as the Laplacian. There is no difference and in light of that, our error budget should be viewed as a between the two estimations of the position of the lower limit on what is occurring in the real world. Such is the discontinuity. price paid undertaking any simulation. Pascal’s triangle of integers can also be thought of as resulting from the repeated smoothing of the initial solitary value of unit intensity by the elementary smoothing kernel [+1, 11. Epilogue +1]. ThusK0, 0, 1, 0, 0,K upon smoothing, becomes K 0, 0, Imagine we have two people approaching each other in the 1, 1, 0, 0, K and then K 0, 0, 1, 2, 1, 0, 0, K and then K 0, 0, fog. They each know that there is a cliff ahead of them, but it is 1, 3, 3, 1, 1, 0, 0, K etc. That sequence is Pascal’s triangle. too dark to see it. One is walking from the sea, approaching the Differentiating any row in Pascal’s triangle (that is, differen- cliff from below. The other is high above the water on a gently cing adjacent numbers in the triangle) is similarly visualized by sloping hill approaching the cliff from above. The ﬁrst may be having the row convolved by the zero-sum differencing kernel noticing that she is walking uphill away from the water, [+1, −1]. Applying the ﬁrst row of Pascal’s triangle gives K navigating undulating sand dunes, etc. None of this topology of 0, 0, −1, +1, 0, 0, K, which is a very compact ﬁrst derivative the local terrain can alert him to the discontinuity that he is of adjacent pixels measured at their interface. An application of walking towardK until he slams into it. The second adventurer the differencing kernel to the second line of Pascal’s triangle notices the cracks and crevasses that he has to walk over or gives K 0, 0, +1, 0, −1, 0, 0, K, which ushers in the around, but again nothing at his feet alerts him of his pending appearance of Sobel ﬁlter, as mentioned above. An application doom K until he walks off the cliff. The AGB is the sandy to the third line givesK 0, 0, −1, −1, +1, +1,K; and then the seaside below. The RGB is the grassy meadow above. Neither fourth line gives K 0, 0, −1, −2, 0, +2, +1, 0, 0, K etc. of those features can predict what lays ahead. Having said this in words, the table in Figure 22 shows the ﬁrst 13 rows of Pascal’s triangle, while the table in Figure 23 shows the ﬁrst 13 rows of the ﬁrst (digital) derivative of Pascal’s Appendix A Digital Filters and Smoothing in Edge-detection Cioni et al. (2000) were the ﬁrst to suggest the use of the Laplacian as a In most prescriptions for edge-detection in digital image means of locating the TRGB. Their approach differed somewhat from what we have discussed above, and they warn users about a potential bias between the processing, it is advised that the raw image be smoothed ﬁrst to Laplacian and the Sobel-ﬁlter solutions. We have been in communication with reduce the random noise in the image and then followed by an Dr. Cioni, and we now all agree that, when using the Laplacian, its zero additional scan of the data using a ﬁrst-derivative edge-detector crossing should be used to identify the discontinuity, and that this measure is that responds to locally detected gradients across the image. not biased with respect to the Sobel ﬁlter. 26 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 21. Binomial coefﬁcient and their ﬁrst derivatives. Each of the panels shows the values of the binomial coefﬁcients, numerically in square brackets and graphically as vertical lines in the upper half of each plot. The smooth curve is a Gaussian. In the lower half of the panel is the ﬁrst derivative of the binomial kernel, again given numerically inside of square brackets, and graphically as vertical black lines bounded by the smooth black line, which is the ﬁrst derivative of the Gaussian. triangle. In Figure 21, we show the ﬁrst four even-numbers sets Gaussian (and its derivative) each has larger support; we are of binomial coefﬁcients (MF3 through MF9) with a smooth spanning more and more pixels and thereby implicitly weight- Gaussian overlaid in the upper panels and symmetrically smoothing the data, in addition to any previous smoothing. In sampling the DoG, including its central point at the zero- Figure 24, we show the application of this single-step crossing point of the kernel, shown in the lower subpanels. methodology to a step function. Here, it is noteworthy that This immediately suggests that, in the process of going to the width of the response is largely independent of the order of higher and higher approximations, the discretely sampled the ﬁlter chosen. 27 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 22. The binomial coefﬁcients of Pascal’s triangle. Each line represents a digital kernel, each of which is a progressively higher-ﬁdelity approximation to a Gaussian. Applied to digital data, these kernels act as weighted smoothing Figure 24. Examples of the Pascal edge-detector response to a step function. functions. These are the results of applying the third, ﬁfth, and seventh (centrally peaked) ﬁlters given in Figure 22 (above), shown as red, blue, and black lines, respectively. They are scaled to equal areas, demonstrating the relatively stable width of the response-functions, independent of the smoothing width of the chosen edge-detector. Appendix B A Closing Comment about the Sensitivity of the Adopted Differencing Kernels to Structure Surrounding the Discontinuity It needs to be said in this closing remark that the discontinuity of the LF (especially as seen directly in the I band, and in the rectiﬁed LFs constructed at other wavelengths) is a very locally deﬁned quantity. By that we mean that only information contained in a handful of milli-magnitude bins, ahead of and/or following the discontinuity itself, contributes to the tipʼs detection. Moreover, the presence or absence of stars farther away from the action (i.e., from the TRGB discontinuity) can have little or no inﬂuence on the output of the edge-detection ﬁlter, since all values outside of the kernel Figure 23. First derivatives of the binomial coefﬁcients in Pascal’s triangle, as and/or ﬁlter’s support are set to zero. For instance, given the given in Figure 23 above. The ﬁrst derivative of a Gaussian (DoG) is a well- ﬁnite range of support adopted by the Sobel ﬁlter (the simplest known edge-detector in image processing. Each of the entries in this ﬁgure are example being [−1, 0, +1]), only those AGB stars that have then also edge-detectors, progressively more precise approximation to the magnitudes that are within plus or minus one bin of the TRGB DoG. Examples of their application to the detection of a step function are shown in Figure 23 (below). (AGB stars above, and RGB stars below) will have any effect 28 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. on the output of the ﬁlter at the tip. The slopes of the respective Figure 12). The two papers both offer a quantitative means of AGB or RGB LFs (be they positive, ﬂat, or negative) will only selecting an optimal smoothing parameter, which is the one that produce a constant output (of the ﬁrst-derivative ﬁlter) up until minimizes the quadrature sum of the random and systematic a signiﬁcant transition in the slope is detected (above and errors. beyond the noise). Poisson noise at the tip will serve to smooth To shed further light on that question, we offer Figure 25, the tip, but it will not bias the tip detection, nor will the ﬁlter which consists of nine subpanels containing repeated runs of care (or know) what is happening to stars more than a tenth of a the simulation displayed in subpanel (e) of Figure 11. All of the magnitude (say) above or below the tip. The presence of AGB input parameters were ﬁxed, and only the random sampling of stars immediately before the tip can only act to change the the LF was allowed to change. Details are given in the contrast in the jump by adding a pillar of counts to the extended ﬁgure caption, but our conclusions are that at this difference being measured between the AGB base (seen at one smoothing the displacements are random, but given the larger side of the differencing kernel) and (the sudden) onset of the density of false (minor) peaks below the tip as compared to RGB (seen by the other side of the advancing kernel). Nothing similar ﬂuctuations being registered above the tip, we warn that else much matters. Everything about the TRGB is local. larger amounts of smoothing will result in systematic shifts of A more compact and mathematically formal way of looking the measured tip to fainter magnitudes. at it is the fact that the derivative of a function (dF/dx) at x(i) is found in the limit as the differencing interval (dx) goes to zero at x(i) . It does not matter what F is doing at x(i + 5),or x(i + Appendix D 10),or x(i − 5),or x(i − 10), etc. Demonstration of the Lack of Bias in the Sobel Tip Detection to Smoothing of a Variety of AGB Luminosity Functions and RGB Stars above and below the TRGB Appendix C Random Displacements of the Tip Revealed in Multiple In Figures 26 through 28, we show the robust nature of the Realizations of a Single Numerical Experiment simple Sobel-ﬁlter response to the the application of smooth- ing, and to three possible forms of the AGB LF approaching At the urgings of the referee, we have investigated whether the TRGB from above. Figure 26 shows a declining AGB LF. the mismatch between the observed and true tip magnitudes is Figure 27 shows an increasing AGB LF, and Figure 28 shows a systematic or random in nature. This experiment has already ﬂat AGB LF as has been adopted in the simulations given in been run and published in the study of M101 by Beaton et al. the main paper. (2019; their Figure 5 and extended caption; and their Figure 8). As expected, given the symmetric nature of the kernels being The latter shows the effect of oversmoothing where highly applied in the smoothing and in the tip detection, there is no smoothed detections drift away from their lesser smoothed resulting bias in the position reported by the Sobel-ﬁlter versions. In addition, a wide range of edge-detection methods response-function. The effects of noise and the Sobel response using different smoothing kernels and even including those using maximum-likelihood ﬁtting techniques agreed when to smoothing is also nicely discussed in Nayar (2022), applied to the same data set for IC 1613 (Hatt et al. 2017; especially his Figures 25–27. 29 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 25. Nine subpanels illustrating a random sampling of TRGB measurements for the same number of RGB stars (1340) the same smoothing (0.10 mag) at the tip (±0.15 mag) as in subpanel (e) in Figure 11. These independently selected examples demonstrate the random drift of the peak response of the Sobel ﬁlter around the input value shown by the solid vertical black line at 0.0 mag. One peak ((b), (h), and (i)). Read left to right and top to bottom, the peak in subpanel (i) is noticeably displaced to a brighter magnitude; the example in subpanel (e) is displaced to fainter magnitudes. Other deﬂections are all within the 1σ expected deviations shown by the vertical dashed lines; ﬁve ((a), (b), (d), (g), and (h)) fall to the left, and two ((c) and (f)) fall to the right, although the latter is ﬂanked by two peaks that are apparently more signiﬁcant than the one found closest to the known answer. We note that more (low-level) structure in the output response is at magnitudes fainter than the true tip. Some of this structure is close enough (subpanels (d) and (i)) that, if excessive smoothing were applied, that action would preferentially draw the measured tip magnitudes systematically toward fainter magnitudes. 30 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. ORCID iDs Barry F. Madore https://orcid.org/0000-0002-1576-1676 Wendy L. Freedman https://orcid.org/0000-0003- 3431-9135 Kayla A. Owens https://orcid.org/0000-0003-3339-8820 In Sung Jang https://orcid.org/0000-0002-2502-0070 References Anand, G. S., Tully, R. B., Rizzi, L., et al. 2022, ApJ, 932, 15 Anderson, R. I., Koblischke, N. W., & Eyer, L. 2023, arXiv:2303.04790 Baade, W. 1944, ApJ, 100, 137 Beaton, R. L., Seibert, M., Hatt, D., et al. 2019, ApJ, 885, 141 Berendzen, R., Hart, R., & Seeley, D. 1976, Man Discovers the Galaxies (New York: Science History Publications), 43 Brown, T. M., Ferguson, H. C., Smith, E., et al. 2003, ApJL, 592, 17 Cassisi, S., & Salaris, M. 2013, Old Stellar Populations (New York: Wiley- VCH), doi:10.1002/9783527665518 Figure 26. Toy model of a rising AGB luminosity function + an exponential Cioni, M.-R. L., van der Marel, R. P., Loup, C., et al. 2000, A&A, 359, 601 RGB luminosity function, Gaussian smoothed (upper subpanes) and convolved Conn, A. R., Lewis, G. F., Ibata, R. A., et al. 2011, ApJ, 740, 69 with a Sobel edge-detection response ﬁlter (lower subpanels). Da Costa, G. S., & Armandroff, T. E. 1990, AJ, 100, 162 Dalcanton, J. J., Williams, B. F., Seth, A. C., et al. 2009, ApJS, 183, 67 Durbin, M. J., Beaton, R. L., Dalcanton, J. J., et al. 2020, ApJ, 898, 57 Eyer, L., Rimoldini, L., Audard, M., et al. 2019, A&A, 623, 110 Freedman, W. L. 1988a, AJ, 96, 1248 Freedman, W. L. 1988b, ApJ, 326, 691 Freedman, W. L. 2021, ApJ, 919, 16 Freedman, W.L., et al. 2023, ApJ, submitted Freedman, W. L., Madore, B. F., Hatt, D., et al. 2019, ApJ, 882, 34 Frogel, J., Cohen, J., & Persson, S. E. 1983, ApJS, 53, 713 Gorski, M., Pietrzynski, G., Gieren, W., et al. 2018, AJ, 156, 278 Hatt, D., Beaton, R. l., Freedman, W. L., et al. 2017, ApJ, 845, 146 Hoyt, T. 2023, arXiv:2106.13337 Hoyt, T., Freedman, W. L., Madore, B. F., et al. 2018, ApJ, 858, 150 Jahne, B. 1991, Digital Image Processing: Concepts, Algorithms and Scientiﬁc Applications (New York: Springer), doi:10.1007/978-3-662-11565-7 Jang, I.-S., Hatt, D., Beaton, R. L., et al. 2018, ApJ, 852, 60 Jang, I.-S., & Lee, M.-G. 2017, ApJ, 835, 28 Lamers, H. G. L. M., & Levesque, E. M. 2017, Understanding Stellar Evolution (Bristol: IOP Publishing) doi:10.1088/978-0-7503-1278-3 Lee, M.G., Freedman, W.L., Madore, B.f., et al. 1993, ApJ, 417, 553 Lee, M.-G., & Jang, I.-S. 2012, ApJ, 760, 14L Madore, B. F., & Freedman, W. L. 1995, AJ, 109, 1645 Figure 27. Toy model of declining AGB luminosity + an exponential RGB Madore, B. F., Freedman, W. L., et al. 2018, ApJ, 858, 11 luminosity function, Gaussian smoothed (upper subpanes) and convolved with Madore, B. F., Mager, V., & Freedman, W. L. 2009, ApJ, 690, 389 a Sobel edge-detection response ﬁlter (lower subpanels). Madore, B. F., McAlary, C. W., McLaren, R. A., et al. 1985, ApJ, 294, 560 Makarov, D., Makarova, L., Rizzi, L., et al. 2006, AJ, 132, 2729 McQuinn, K. B. W., Boyer, M., Skillman, E. D., et al. 2019, ApJ, 880, 63 Menendez, B., Davis, M., & Moustakas, J. 2002, AJ, 124, 213 Monachesi, A., Bell, E. F., Radburn-Smith, D. J., et al. 2013, ApJ, 780, 179 Mould, J., & Kristian, J. 1986, ApJ, 305, 591 Mould, J., Kristian, J., & Da Costa, G. S. 1983, ApJ, 270, 471 Mould, J., Kristian, J., & Da Costa, G. S. 1984, ApJ, 278, 575 Nayar, S. 2022, Monograph: FPCV-2-1, Series: First Principles of Computer Vision, Computer Science (New York: Columbia Univ.), https://cave.cs. columbia.edu/Statics/monographs/Edge%20Detection%20FPCV-2-1.pdf Nikolaev, S., & Weinberg, M. D. 2000, ApJ, 542, 804 Rizzi, L., Tully, R. B., Makarov, D., et al. 2007, ApJ, 661, 815 Salaris, M., & Cassisi, S. 2005, Evolution of Stars and Stellar Populations (New York: Wiley) Sandage, A. R., & Carlson, G. 1983, ApJL, 267, 25 Scolnic, D., Riess, A.G., Wu, S., et al. 2023, arXiv:2304.06693 Serenelli, A., Weiss, A., Cassisi, S., et al. 2017, A&A, 606, 33 Shapley, H. 1918, ApJ, 48, 89 Shapley, H. 1919, ApJ, 49, 311 Shapley, H. 1930, Star Clusters (London: McGraw-Hill) Silverman, B. W. 1986, Density Estimation for Statistics and Data Analysis Figure 28. Toy model of a ﬂat AGB luminosity function + an exponential (London: Chapman and Hall) RGB luminosity function, Gaussian smoothed (upper subpanes) and convolved Soszynski, I., Udalski, A., Kubiak, M., et al. 2004, AcA, 54, 129 with a Sobel edge-detection response ﬁlter (lower subpanels). Wu, P.-F., Tully, R. B., Rizzi, L., et al. 2014, AJ, 148, 7 http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png The Astronomical Journal IOP Publishing http://www.deepdyve.com/lp/iop-publishing/quantifying-uncertainties-on-the-tip-of-the-red-giant-branch-method-h0xNnwymPk

Loading next page...

References (42)

M. Lee, W. Freedman, B. Madore (1993)
The Tip of the Red Giant Branch as a Distance Indicator for Resolved Galaxies
The Astrophysical Journal, 417
M. Durbin, R. Beaton, J. Dalcanton, B. Williams, M. Boyer (2020)
MCR-TRGB: A Multiwavelength-covariant, Robust Tip of the Red Giant Branch Measurement Method
The Astrophysical Journal, 898
R. Beaton, M. Seibert, D. Hatt, W. Freedman, Taylor Hoyt, I. Jang, M. Lee, B. Madore, A. Monson, J. Neeley, J. Rich, V. Scowcroft (2019)
The Carnegie-Chicago Hubble Program. VII. The Distance to M101 via the Optical Tip of the Red Giant Branch Method
The Astrophysical Journal, 885
L. Eyer, L. Rimoldini, M. Audard, R. Anderson, K. Nienartowicz, F. Glass, O. Marchal, M. Grenon, N. Mowlavi, B. Holl, G. Clementini, C. Aerts, T. Mazeh, D. Evans, L. Szabados, A. Brown, A. Vallenari, T. Prusti, J. Bruijne, C. Babusiaux, C. Bailer-Jones, M. Biermann, F. Jansen, C. Jordi, S. Klioner, U. Lammers, L. Lindegren, X. Luri, F. Mignard, C. Panem, D. Pourbaix, S. Randich, P. Sartoretti, H. Siddiqui, C. Soubiran, F. Leeuwen, N. Walton, Frédéric Arenou, U. Bastian, M. Cropper, R. Drimmel, D. Katz, M. Lattanzi, J. Bakker, C. Cacciari, J. Castañeda, L. Chaoul, N. Cheek, F. Angeli, C. Fabricius, R. Guerra, E. Masana, R. Messineo, P. Panuzzo, J. Portell, M. Riello, G. Seabroke, P. Tanga, F. Thévenin, G. Gracia-Abril, G. Comoretto, M. Garcia-Reinaldos, D. Teyssier, M. Altmann, R. Andrae, I. Bellas-Velidis, K. Benson, J. Berthier, R. Blomme, P. Burgess, G. Busso, B. Carry, A. Cellino, M. Clotet, O. Creevey, M. Davidson, J. Ridder, L. Delchambre, A. Dell’Oro, C. Ducourant, J. Fernández-Hernández, M. Fouesneau, Y. Frémat, L. Galluccio, M. García-Torres, J. González-Núñez, J. González-Vidal, E. Gosset, L. Guy, J. Halbwachs, N. Hambly, D. Harrison, J. Hernández, D. Hestroffer, S. Hodgkin, A. Hutton, G. Jasniewicz, A. Jean-Antoine-Piccolo, S. Jordan, A. Korn, A. Krone-Martins, A. Lanzafame, T. Lebzelter, W. Löffler, M. Manteiga, P. Marrese, J. Martín-Fleitas, A. Moitinho, A. Mora, K. Muinonen, J. Osinde, E. Pancino, T. Pauwels, J. Petit, A. Recio-Blanco, P. Richards, A. Robin, L. Sarro, C. Siopis, M. Smith, A. Sozzetti, M. Süveges, J. Torra, W. Reeven, U. Abbas, A. Aramburu, S. Accart, G. Altavilla, M. Álvarez, R. Álvarez, J. Alves, A. Andrei, E. Varela, E. Antiche, T. Antoja (2018)
Gaia Data Release 2: Variable stars in the colour-absolute magnitude diagram
arXiv: Solar and Stellar Astrophysics
B. Madore, W. Freedman (1995)
The tip of the Red Giant Branch as a Distance Indicator for Resolved Galaxies: II. Computer Simulations
The Astronomical Journal, 109
J. Frogel, S. Persson, J. Cohen (1983)
Infrared photometry, bolometric luminosities and effective temperatures for giant stars in 26 globular clusters.
Astrophysical Journal Supplement Series, 53
H. Lamers, E. Levesque (2018)
Understanding Stellar Evolution
W. Freedman, B. Madore, D. Hatt, Taylor Hoyt, I. Jang, R. Beaton, C. Burns, M. Lee, A. Monson, J. Neeley, M. Phillips, J. Rich, M. Seibert (2019)
The Carnegie-Chicago Hubble Program. VIII. An Independent Determination of the Hubble Constant Based on the Tip of the Red Giant Branch
The Astrophysical Journal, 882
K. McQuinn, M. Boyer, E. Skillman, A. Dolphin (2019)
Using the Tip of the Red Giant Branch As a Distance Indicator in the Near Infrared
The Astrophysical Journal, 880
W. Freedman (1988)
Stellar content of nearby galaxies. I - BVRI CCD photometry for IC 1613
The Astronomical Journal, 96
B. Madore, C. Mcalary, R. Mclaren, D. Welch, G. Neugebauer, K. Matthews (1985)
Near-Infrared Observations of Cepheids: The Distance to NGC 300
The Astrophysical Journal, 294
Bernard Silverman (1987)
Density Estimation for Statistics and Data Analysis
(2003)
TO APPEAR IN THE ASTROPHYSICAL JOURNAL LETTERS Preprint typeset using L ATEX style emulateapj v. 04/03/99 EVIDENCE FOR A SIGNIFICANT INTERMEDIATE-AGE POPULATION IN THE M31 HALO FROM
W. Baade (1944)
The Resolution of Messier 32, NGC 205, and the Central Region of the Andromeda Nebula
The Astrophysical Journal, 100
J. Mould, J. Kristian (1986)
The Stellar Population in the Halos of M31 and M33
The Astrophysical Journal, 305
(2012)
ApJ, 760,14L Madore, B.F
M. Salaris, S. Cassisi (2005)
Evolution of Stars and Stellar Populations
Galactic Astronomy
H. Shapley (1919)
Studies based on the colors and magnitudes in stellar clusters. XII. Remarks on the arrangement of the sidereal universe.
The Astrophysical Journal, 49
G. Anand, R. Tully, L. Rizzi, A. Riess, W. Yuan (2021)
Comparing Tip of the Red Giant Branch Distance Scales: An Independent Reduction of the Carnegie-Chicago Hubble Program and the Value of the Hubble Constant
The Astrophysical Journal, 932
B. Madore, W. Freedman, D. Hatt, Taylor Hoyt, A. Monson, R. Beaton, J. Rich, I. Jang, M. Lee, V. Scowcroft, M. Seibert (2018)
The Near-infrared Tip of the Red Giant Branch. I. A Calibration in the Isolated Dwarf Galaxy IC 1613
The Astrophysical Journal, 858
Taylor Hoyt, W. Freedman, B. Madore, M. Seibert, R. Beaton, D. Hatt, I. Jang, M. Lee, A. Monson, J. Rich (2018)
The Near-infrared Tip of the Red Giant Branch. II. An Absolute Calibration in the Large Magellanic Cloud
The Astrophysical Journal, 858
B. Mendez, M. Davis, J. Moustakas, J. Newman, B. Madore, W. Freedman (2002)
Deviations from the Local Hubble Flow. I. The Tip of the Red Giant Branch as a Distance Indicator
The Astronomical Journal, 124
R. Berendzen, R. Hart, D. Seeley, J. Day (1977)
Man discovers the galaxies
The Physics Teacher, 16
(2013)
Monograph: FPCV-2-1, Series: First Principles of Computer
A. Serenelli, A. Weiss, S. Cassisi, M. Salaris, Adriano Sciences, M. Astrophysics, I. Teramo, A. Institute, Liverpool University (2017)
The brightness of the Red Giant Branch tip: Theoretical framework, a set of reference models, and predicted observables
arXiv: Solar and Stellar Astrophysics
Po-Feng Wu, R. Tully, L. Rizzi, A. Dolphin, B. Jacobs, I. Karachentsev (2014)
INFRARED TIP OF THE RED GIANT BRANCH AND DISTANCES TO THE MAFFEI/IC 342 GROUP
The Astronomical Journal, 148
W. Freedman, B. Madore, S. Hawley, I. Horowitz, J. Mould, M. Navarrete, S. Sallmen (1988)
New Cepheid distances to nearby galaxies based on BVRI CCD photometry. III: NGC 300
The Astrophysical Journal, 326
D. Brouwer (1931)
Star Clusters . (Harvard Observatory Monographs No. 2). By Harlow Shapley. xi + 276 pp. McGraw- Hill Book Company, New York, 1930. $3.00.
Science
(2000)
were the first to suggest the use of the Laplacian as a means of locating the TRGB
S. Nikolaev, M. Weinberg (2000)
Stellar Populations in the Large Magellanic Cloud from 2MASS
The Astrophysical Journal, 542
I. Jang, M. Lee (2016)
THE TIP OF THE RED GIANT BRANCH DISTANCES TO TYPE IA SUPERNOVA HOST GALAXIES. IV. COLOR DEPENDENCE AND ZERO-POINT CALIBRATION
The Astrophysical Journal, 835
I. Jang, D. Hatt, R. Beaton, M. Lee, W. Freedman, B. Madore, Taylor Hoyt, A. Monson, J. Rich, V. Scowcroft, M. Seibert (2017)
The Carnegie–Chicago Hubble Program. III. The Distance to NGC 1365 via the Tip of the Red Giant Branch
The Astrophysical Journal, 852
J. Mould, J. Kristian, G. Costa (1983)
Stellar populations in Local Group dwarf elliptical galaxies. I: NGC 147
The Astrophysical Journal, 270
H. Shapley (1918)
Studies based on the colors and magnitudes in stellar clusters. VI. On the determination of the distances of globular clusters.
The Astrophysical Journal, 48
A. Sandage, G. Carlson (1983)
The distance to M 33 based on a new study of its Cepheids.
The Astrophysical Journal, 267
D. Hatt, R. Beaton, W. Freedman, B. Madore, I. Jang, Taylor Hoyt, M. Lee, A. Monson, J. Rich, V. Scowcroft, M. Seibert (2017)
The Carnegie-Chicago Hubble Program. II. The Distance to IC 1613: The Tip of the Red Giant Branch and RR Lyrae Period–luminosity Relations
The Astrophysical Journal, 845
M. G'orski, G. Pietrzy'nski, W. Gieren, D. Graczyk, K. Suchomska, P. Karczmarek, R. Cohen, B. Zgirski, P. Wielg'orski, B. Pilecki, M. Taormina, Z. Kołaczkowski, W. Narloch (2018)
The Araucaria Project: Multi-band Calibrations of the TRGB Absolute Magnitude
The Astronomical Journal, 156
J. Dalcanton, B. Williams, A. Seth, A. Dolphin, J. Holtzman, K. Rosema, E. Skillman, A. Cole, L. Girardi, S. Gogarten, I. Karachentsev, K. Olsen, D. Weisz, C. Christensen, K. Freeman, K. Gilbert, C. Gallart, J. Harris, P. Hodge, R. Jong, V. Karachentseva, M. Mateo, P. Stetson, M. Tavarez, D. Zaritsky, F. Governato, T. Quinn (2007)
THE ACS NEARBY GALAXY SURVEY TREASURY
The Astrophysical Journal Supplement Series, 183
(1993)
Evolution (Bristol: IOP Publishing
A. Monachesi, E. Bell, D. Radburn-Smith, R. Jong, J. Bailin, J. Dalcanton, B. Holwerda, H. Ford, D. Streich, M. Vlajić, D. Zucker (2013)
GHOSTS I: A NEW FAINT VERY ISOLATED DWARF GALAXY AT D = 12 ± 2 Mpc
The Astrophysical Journal, 780
W. Freedman (2021)
Measurements of the Hubble Constant: Tensions in Perspective
The Astrophysical Journal, 919
G. Costa, T. Armandroff (1990)
Standard Globular Cluster Giant Branches in the (M(I), (V - I)o) Plane
The Astronomical Journal, 100

Publisher: IOP Publishing
Copyright: © 2023. The Author(s). Published by the American Astronomical Society.
ISSN: 0004-6256
eISSN: 1538-3881
DOI: 10.3847/1538-3881/acd3f3
Publisher site: See Article on Publisher Site

Abstract

We present an extensive grid of numerical simulations quantifying the uncertainties in measurements of the tip of the red giant branch (TRGB). These simulations incorporate a luminosity function composed of 2 mag of red giant branch (RGB) stars leading up to the tip, with asymptotic giant branch (AGB) stars contributing exclusively to the luminosity function for at least a magnitude above the RGB tip. We quantify the sensitivity of the TRGB detection and measurement to three important error sources: (1) the sample size of stars near the tip, (2) the photometric measurement uncertainties at the tip, and (3) the degree of self-crowding of the RGB population. The self- crowding creates a population of supra-TRGB stars due to the blending of one or more RGB stars just below the tip. This last population is ultimately difﬁcult, although still possible, to disentangle from true AGB stars. In the analysis given here, the precepts and general methodology as used in the Chicago-Carnegie Hubble Program (CCHP) have been followed. However, in the appendix, we introduce and test a set of new tip detection kernels, which internally incorporate self-consistent smoothing. These are generalizations of the two-step model used by the CCHP (smoothing followed by Sobel-ﬁlter tip detection), where the new kernels are based on successive binomial-coefﬁcient approximations to the derivative-of-a-Gaussian edge-detector, as is commonly used in modern digital image processing. Uniﬁed Astronomy Thesaurus concepts: Red giant stars (1372); Distance indicators (394) 1. Introduction nearly a full magnitude closer at (m − M) = 24.25 ± 0.15 mag. The TRGB distance fell in the mid-range, at (m − M) = Over a century ago, Shapley (1918, 1919, 1930) used blue- 24.8 ± 0.2 mag, right between the other two extremes. How- sensitive photographic plates to measure (by eye) the mean ever, not all of the early cross-comparisons of TRGB and apparent magnitudes of the 25 brightest stars in galactic Cepheid distance scales were in conﬂict. For example, globular clusters (his Table 1, 1919), in order to go on to Freedman (1988a) used the ﬁrst CCD camera available on (incorrectly) build a case for his version of an Island Universe the CFHT and measure the TRGB in the halo of the Local cosmology (see Berendzen et al. 1976). With the availability of Group dwarf irregular galaxy, IC 1613. She found a true newly developed, red-sensitive photographic plates, Walter distance modulus of (m − M) = 24.2 mag, which did agree the Baade (1944) serendipitously resolved the brightest red giant Cepheid-based distance modulus of (m − M) = 24.3 mag stars (which, to his surprise, suddenly appeared at approxi- (Freedman 1988b). mately the same red-band magnitudes) in several dwarf The TRGB method ﬁnally came of age with the publication elliptical companions galaxies to the Andromeda galaxy, of two papers: the ﬁrst was the calibration paper by Da Costa & M31. That unanticipated discovery precipitated a revision in Armandroff (1990) who were inspired to undertake an I-band the size and age of the universe by a factor of 2. Four decades CCD survey of a sample of 8 southern Milky Way globular later, and armed with some of the ﬁrst available panoramic clusters. In doing so, they demonstrated that, while the mean linear charge-coupled devices (CCDs), Mould et al. (1984) colors of the giant branches were rank-ordered by the mean revisited one of Baade’s original dwarf galaxies, NGC 205. metallicities of the parent globular clusters (as previously They produced full color–magnitude diagrams (CMDs) known to Frogel et al. 1983 from pioneering their studies of revealing a broad swath of red giant branch (RGB) stars all RGB stars in globular clusters in the near infrared), the of which cumulatively deﬁned a constant I-magnitude plateau brightest of those RGB stars had a remarkably stable absolute in the CMD, later to be named the tip of the red giant branch, or magnitude, in the I band, independent of color. The second simply known by its initialism, the TRGB. They also had paper was that of Lee et al. (1993). It laid out, in one place, earlier observed NGC 147 (Mould et al. 1983) ﬁnding the same most of the key issues concerning systematics involving feature. But perhaps more interestingly, they (Mould & reddening, metallicity, star formation history, and host galaxy Kristian 1986) observed TRGB stars in the halo of the Local type, etc. It also introduced the widely adopted Sobel ﬁlter for Group spiral galaxy, M33. By good fortune, at about the same precisely deriving the magnitude at which the discontinuity in time, M33 had been the subject of two different investigations the RGB luminosity function (LF) occurs, as well as its into Cepheid distance moduli to this galaxy: one by Sandage & uncertainty, while exploring a range of smoothing kernels. This Carlson (1983) coming in high, with a value of was carried out in the context of anticipating a refurbished (m − M) = 25.23 mag; and another by Madore et al. (1985) Hubble Space Telescope, and applying the TRGB method widely to the extragalactic distance scale. The authors Original content from this work may be used under the terms demonstrated its ground-based application to 10 galaxies of the Creative Commons Attribution 4.0 licence. Any further spanning a wide range of Hubble types, metallicities, and distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI. absolute magnitudes, and found overall consistency in the 1 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. TRGB, Cepheid, and RR Lyrae distance scales at the level of populated individual galactic globular clusters, or very small 0.1 mag. The success of the TRGB method might be measured ﬁelds of view in the halos of very nearby galaxies, say). The by its subsequent adoption: over the intervening three decades, ﬁeld has matured, the demand for higher precision has more than 500 TRGB distances to nearby galaxies have been prevailed, and the numbers of stars measured in extragalactic published (for instance, NED lists over 900 references to halo ﬁelds have gone into the thousands, while at the same time TRGB distance determinations to 302 distinct galaxies; and other sources of uncertainty in determining the precision of the EDD lists 588 galaxies that they have derived uniformly TRGB have become clear. We feel that it is time now to processed TRGB distances ). And recently, the TRGB method explore parameter space a bit more thoroughly. In the has been extended to the calibration of Type Ia supernovae and following, we consider, in turn, a total of three independent, determination of the Hubble constant (Freedman et al. 2019; major sources of uncertainty: Freedman 2021). (1) The formal way in which the statistical uncertainty in the For theoretical discussions of the evolution of stars up to and tip magnitude can be quantiﬁed, speciﬁcally in terms of including degenerate helium core ﬂash, we recommend that its sensitivity to numbers of stars at the tip, and its readers consult the monographs by Cassisi & Salaris (2013), independent sensitivity to individual photometric errors Salaris & Cassisi (2005), and Lamers & Levesque (2017). For of those same tip-deﬁning stars. updated discussions of modeling, with special reference to (2) The effects of having an asymptotic giant branch (AGB) near- and mid-infrared applications of the TRGB method, see population of stars contributing to the 1 mag interval Serenelli et al. (2017), McQuinn et al. (2019), and Durbin directly above the TRGB. et al. (2020). (3) And ﬁnally, the explicit modeling of the mutual (line-of- sight) crowding of all stars along the RGB, and the 2. Motivation inevitable production of a new, but totally spurious, population of (crowded) stars, systematically brighter A quick census of the published determinations of the than the TRGB. apparent magnitude of the TRGB in even the nearest of galaxies (NED-D 2022 August version) immediately reveals a We use a modiﬁed Sobel edge-detection ﬁlter (see wide range of quoted uncertainties. The published errors, for Appendix B) for measurement of the TRGB, which is largely the tip measurement in a given galaxy, can vary by as much as consistent with our GLOESS-smoothed, Sobel-ﬁltered analysis a factor of 10; as in the case of M31 (0.05–0.57 mag used in the CarnegieChicago Hubble Program (Hatt et al. 2017; uncertainties quoted) and M33 (0.03–0.30 mag); but more Hoyt et al. 2018; Jang et al. 2018; Madore et al. 2018).In typically, they range over a factor of 3–6 as, for example, in the Appendix B, we derive and tabulate a complete series of new published values for the nearest galaxies: the LMC digital ﬁlters that are derived from successive discrete (0.04–0.25 mag), IC 1613 (0.05–0.20 mag), and NGC 6822 approximations of the ﬁrst derivative of a Gaussian (DoG), (0.06–0.19 mag). On the other hand, some of the reported using the binomial theorem as the gradient detector. We also statistical uncertainties on the tip determination can go as low adopt the weighted (noise-suppression) versions of these as 0.01 mag (e.g., Lee & Jang 2012 for Messier 101 (M101);or kernels as ﬁrst introduced and applied to a simple Sobel ﬁlter even smaller than that in the case of Conn et al. 2011 for in Madore et al. (2009) and much later adopted and utilized by Andromedas I and II). In an appendix to Cioni et al. (2000), Gorski et al. (2018). those authors rightly note that many of the methods used, (counterintuitively) do not in any way scale with population 3. The Underlying Model size of stars detected and measured at the tip. They should. For The basic model adopted here for the intrinsic LF, above and a given photometric error, population size certainly needs to be below the TRGB, now consists of three distinct input a part of the calculation of the statistical uncertainty on the populations: (a) a RGB population with a power-law increase mean of the TRGB distance. Upon closer examination of any in numbers with increasing (fainter) magnitudes, (b) a bright- given paper, it is not always clear what exactly the source of the end truncation/discontinuity of the RGB LF, deﬁning the tip, quoted uncertainty is or even how it was actually calculated. In and (c) an AGB population, stretching at least 1 mag above and this paper, we attempt to bring some clarity to the situation. brighter than the TRGB. We model the LF from 1 mag above In earlier papers (Madore & Freedman 1995; Madore to 2 mag below the TRGB (but note that only the ﬁrst et al. 2009), we presented computer simulations of the TRGB magnitude below the tip is shown in the ﬁgures) assuming a ﬂat in its use as an extragalactic distance indicator. In the ﬁrst LF for the AGB down to the TRGB, at which point there is a paper, there is an often quoted and paraphrased conclusion that discontinuous offset to the RGB population. The RGB then “at least 100 stars in the ﬁrst magnitude interval below the tip assumes a steeply rising LF of the form log[( Nm)]= are needed to secure a distance modulus to better than ±0.1 0.3´-[] II +a. In these ﬁrst simulations, the relative TRGB mag.” At that time, the method was still in its infancy, and RGB-to-AGB normalization is six to one, such that there are 17 small number statistics were a major concern (especially when AGB stars in total in the 1 mag interval seen above the TRGB, the early focus was on applying the method to sparsely for every 100 RGB stars in the 1 mag interval fainter than (i.e., below) the tip. For the purpose of this simulation, the AGB LF https://ned.ipac.caltech.edu https://edd.ifa.hawaii.edu/dsecond.php That said, the errors presented in NED are in no way homogenized. NED The referee has argued that a variety of shapes to the AGB luminosity is presents the data as published, and in many cases, the original authors make no apparent in published CMDs including data above the TRGB, and that a ﬂat distinction between statistical and systematic error, or combinations of the two. AGB LF may not be representative. We agree with that statement, but as shown However, see Menendez et al. (2002) and/or Makarov et al. (2006) for in Appendix D, the shape of the AGB LF, be it falling rising or ﬂat, has no extensive discussions speciﬁcally concerning the maximum-likelihood techni- impact on the ability of the Sobel ﬁlter to detect the TRGB in an unbiased que and its error sensitivity to photometry and sample size. manner. 2 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. magnitudes) of TRGB. The lower panel shows the result of running a Sobel ﬁlter [−1, 0, +1] across the binned LF. The output of the Sobel ﬁlter is the discretely sample ﬁrst derivative of the function being sampled. Moving from left to right, the output of the Sobel ﬁlter is constant, as expected, given the constant slope of the input AGB luminosity, i.e., at the ﬁrst position, the output of the Sobel function is [−1 × (N − 3ò) + 0 × 2ò + 1 × (N − 1ò)=] 2ò, where ò is the width of the binning. At the second position, the output is [−1 × (N − 2ò) + 0 × (N − ò) + 1 × N)=] again (the slope of the pure AGB) 2ò. These ﬁrst steps do not sample the discontinuity and therefore contain no information about its position or presence. At step No. 3, the right-most element [+1] is the ﬁrst to sample the discontinuity and reports a increased value of the ﬁlter’s output, [−1 × (N + ò) + 0 × N + 1 × (6/2)N=] 2N − ò. The next step over continues to report larger values of its response-function, where the [+1] element now sees the undiluted height of the RGB LF 6N, and differences that against the response of the left-most element [−1] of the Sobel ﬁlter contributing a value of −N, with the central element of the Sobel ﬁlter always reporting a null value regardless of the function’s value. The output is [−1 × N + 0 × (6/2)N + 1 × 6N=] +5N. The central element simply keeps track of the bin around which the derivative is being measured and reported. Moving the ﬁlter one more bin to the right reports a value of [−1 × (6/2)N + 0 × 6N + 1 × (6N+δ)=] +3N + δ. One step more away from the discontinuity gives [−1 × 6N + 0 × (6N+δ) +1 × (6N +2δ)=] +2δ, the slope of the RGB. All subsequent steps to the right continue to report the constant slope of +2δ. The Figure 1. Magniﬁed view of the idealized toy model of the RGB + AGB luminosity function, centered on the discontinuity in the RGB luminosity maximum value of 5N for the response-function is found at function at the TRGB. Solid yellow segments, from left to right, show the AGB step No. 4 and marks the magnitude at which the TRGB is to LF, the discontinuity, and the RGB LF. The blue histogram is a binned version be found. of straight lines used as digital input to the differencing (Sobel) kernel: [−1, 0, +1]. The digital output of the Sobel response-function is shown in red histogram form in the lower portion of the ﬁgure. The maximum of the Sobel 4. A Few Preliminaries ﬁlter marks the position of the discontinuity. See text for a step-by-step description of the tip detection. It is worth making explicit what exactly the criteria are for a successful experiment to be run, which aims for a detection and is assumed to be ﬂat in the 1 mag interval above the tip and measurement of the position of the discontinuity marking the zero elsewhere. References to the literature justifying the values TRGB in magnitude space. It is then also important to list the for the parameters alluded to above are to be found in the ﬁrst real-world parameters over which we have some control in paragraph of Section 5.1.1. optimally undertaking the observations and subsequently Here we ﬁrst examine an idealization in the form of a toy analyzing the results. model that captures the essential ingredients of the detailed Generally speaking, there are two obvious performance simulations that follow, and try to emphasize how the various indices in TRGB edge-detection that we are concerned with components contribute (or not) to the determination of the here: accuracy and precision. However, the latter (which can magnitude and location of the TRGB discontinuity. The toy also be classiﬁed as bias) can be further broken down into (a) model is shown in Figure 1, an LF centered on the TRGB. This false-positive detections of the TRGB, (b) nondetections, and a plot of logarithm of numbers of stars per magnitude bin as a (c) systematic bias attributed to the edge-detector itself. Each of function of magnitude. The LF is composed of an AGB these are discussed in turn, below. And in the subsection population, represented by a dispersionless straight line sloping following this, we discuss what control we have, at the upward from left to right, stopping one bin short of the location observational design level, in mitigating each of these kinds of of the discontinuity deﬁning the TRGB. The number of AGB errors. stars in that ﬁnal bin is N. One bin beyond that magnitude, the LF is deﬁned by RGB stars whose slope is independent of, and 4.1. Accuracy different from, the AGB slope. The RGB LF is normalized at the tip with a value that is 6 times the value of the AGB (i) False positives. In the presence of random noise in the population (i.e., 6N stars) at its starting point one bin brighter output of our TRGB edge-detection response-function, there than the bin marking the discontinuity. The bin between the comes a point at which (a) the ﬂuctuations in the number of two terminal points deﬁnes the tip, and its value is the average detected stars (from bin to bin) and/or (b) Poisson noise in the of the two adjacent LFs (i.e., 3N stars). photometry of the individual stars themselves will produce The upper panel of Figure 1 shows the input LF binned into (spurious) features in the tip-detection and/or response- 9 histogram-like segments with bin No. 5 centered on the function output. These noise-induced features, if large enough, position of the discontinuity, marking the luminosity (in can be both qualitatively and quantitatively indistinguishable 3 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. from the expected signal (i.e., being positive deﬂections in the progressively increasing the photometric errors (at the tip) response-function, which have a similar width and relative from ±0.02, to 0.05, 0.10, 0.15, and 0.20 mag, respectively height when compared to the expected and/or true signal).In (corresponding to signal-to-noise ratios of 50, 20, 10, 6, and 5). the controlled simulations, discussed below, we quantify when In Figures 3–5, we then rerun parallel simulations, and where this situation starts to become a serious problem. progressively dropping the total population of stars by about (ii) Nondetections. Again, in the presence of excessive a factor of 10 each time: starting with about 120,000 RGB stars photometric noise, in particular, for small population sizes (or in the 1 mag interval below the tip (in Figure 2), and in a combination of the two), it is possible for the true signal to ending with a simulation having only 127 RGB stars in that become so weak that it is not detected at any signiﬁcant same 1 mag interval (in Figure 5). thresholding level, with respect to the ambient response-function In each of the next sections (each also containing four ﬁgures noise. This situation is fatal; but the circumstances under which and six main subpanels), we explore the effects of changing the it is likely to occur can be anticipated and identiﬁed using these smoothing (going up from 0.01 to 0.05, and ﬁnally 0.10 mag) simulations as a guide (see, for example, Figures 4 and 7). in Figures 6 through 9,at ﬁxed population sizes per ﬁgure and (iii) Potential bias in the tip detection algorithm. Given the increasing photometric errors through each of the subpanels, as unequal count rates of stars contributing to the LFs above and in the previous section. below the TRGB, it might be thought that even a symmetric We then close out in Section (4.1) holding the smoothing at a response-function kernel might return an asymmetric (i.e., ﬁxed value (at 0.10 mag) and assessing the effects of changing biased) answer, given that more RGB stars are moving across the population size in Figures 10 through 13, while changing the TRGB discontinuity to intrinsically brighter magnitudes the photometric errors in the subpanels within those ﬁgures. than there are bright AGB stars moving in the opposite This extensive grid of plots is provided both for their use as direction (across the TRGB discontinuity) to fainter magni- predictors in planning future observations, and for their use as a tudes. We investigate this potential source of systematic error guide in understanding the LFs and edge-detector output once (detector bias) throughout the simulations studied below. they are acquired. To put this into perspective for the 12 galaxies observed by Freedman et al. (2019) in their determination of a TRGB-based value of the Hubble constant, 4.2. Precision they detected an average of 4000 RGB stars in the 1 mag We are endeavoring to (a) measure the tip magnitude, (b) interval below the TRGB (with anywhere from 1000 to 20,000 measure its statistical uncertainty (its precision), and (c) RGB stars in individual cases, depending on the distance provide any estimate of bias (its accuracy) inherent in the modulus of the host galaxy and how far into the halo any given methodology explored here. A number of factors contribute to exposure was taken). As for the typical photometric errors at the outcome. Some of these factors can be controlled in the tip, the exposures were scaled to the approximately known advance while setting up the experiment and/or observation, distances, and they all have uncertainties at the tip of and some of them can be ameliorated later in the data analysis about ±0.10 in F814W (I band). This would roughly stage. For instance, the source-count population, the amount of correspond to the middle right panels of Figures 6–8. crowding, and the signal-to-noise ratio in the photometry can To help navigate the various simulations, we provide a guide each be controlled with foreknowledge of the approximate to their ordered content in Table 1. surface brightness of the region being targeted, knowing the approximate distance, and adjusting the total exposure time (or 5.1. Low Degree of Smoothing size of the telescope), within allowable and practical limits. The type of kernel employed in measuring the ﬁrst derivative of the 5.1.1. A Range of Photometric Errors: 120,000 RGB Stars, Fixed LF at and around the tip, and the amount of smoothing of the Smoothing ±0.01 mag data chosen to be applied to the data, in advance of the kernel response-function application, can both be controlled to We start this detailed discussion with a high-deﬁnition optimize the output of the detector once the data have been simulation of the LF beginning 1 mag above, and ending 1 obtained. We consider each of these parameters in turn. mag below the TRGB, where the discontinuity is set to M = 0.00 mag across of the simulations in this paper. This would correspond to M = −4.05 mag, which closely matches the value 5. RGB + AGB Computer Simulations currently adopted by the Chicago-Carnegie Hubble Program In this series of simulations, we explore changing a number (CCHP; Freedman 2021).The ﬁrst magnitude interval, above of parameters while holding others ﬁxed. These include the the tip, is populated uniformly as a function of magnitude by photometric errors and overall population size (Section 4.1), AGB stars. For examples of published ﬂat AGB LFs above the different smoothing sizes (Section 4.2), and amount of tip, see Beaton et al. (2019), their Figure 4, Hoyt et al. (2018), crowding and/or blending (Section 6). In Section 5,we their Figure 6, and Nikolaev & Weinberg (2000), the inset illustrate how the width of the smoothing function does not histogram to their Figure 4, and their description of it being carry information on the uncertainty of the tip measurement. “The off-bar LF shows only a mild increase in the source counts Here, we explore the systematics of changing the photo- at the location of TRGB, but has the same, roughly constant metric errors at the tip (from one simulation to the next) while proﬁle at Ks brighter than 12 mag, due to the AGB population, holding the population size and smoothing ﬁxed. visible in the other two luminosity functions.” At the TRGB With Figure 2, we start at one extreme: a very densely discontinuity, the RGB population turns on at an initial rate (of populated LF (about 120,000 stars in total) having minimal stars per magnitude bin) 6 times greater than the AGB density (0.01 mag) smoothing and very high-precision photometry, as above the tip (see Scolnic et al. 2023, where they calculate a shown in the ﬁrst (upper left) panel. We then work (left to right variant of this contrast ratio R, using bins 0.5 mag wide, above and top to bottom) through the observed effects of and below the tip, for a large number of GHOSTS galaxies, 4 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 2. Six subpanels illustrating the effect of increasing the photometric noise, (from 0.00 to ±0.20 mag) at ﬁxed smoothing (±0.01 mag) and extremely large populations of RGB stars (about 120,000). The lower portions of each of the six subpanels show the ﬁrst-derivative edge-detector output in both its uncorrected (thin black lines) and its noise-weighted (thicker black line) forms. See text for a detailed discussion of the trends. ﬁnding that it ranges from R = 4 to 7 as seen by the annotations is a GLOESS ﬁt with a Gaussian smoothing window of in their Figure 5). Thereafter, the binned number density of RGB 0.01 mag, making it a close approximation, at this ﬁne binning stars increases with a logarithmic slope of +0.3 (Menendez and/or smoothing, to a spline ﬁt through the individual data et al. 2002; Makarov et al. 2006). points. The vertical line at M = 0.0 mag marks the exact The upper left panel in Figure 2 shows our highest-ﬁdelity, position of the TRGB that is equidistantly ﬂanked, in the lower and most optimistic realization, consisting of 120,000 RGB panel, by two dashed lines (barely visible in this panel) that stars and some 20,000 AGB stars. The bin size is 0.01 mag, are ±0.01 mag apart, showing the highest attainable resolution giving a typical RGB population of 1200 stars per bin, leading of the data and the response-function. to an expected 2σ scatter of ±70 stars per bin (or ±6% 1σ,as Below the LF, in the lower part of the panel, is the ﬁrst- can be seen in the plot). The solid line passing through the data derivative response-function as applied to the discretely 5 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 3. Six subpanels illustrating the effect of increasing the photometric noise, (from 0.00 to ±0.20 mag) at ﬁxed smoothing (±0.01 mag) and moderately large populations of RGB stars (11,000). The lower portions of each of the six subpanels show the ﬁrst-derivative edge-detector output in both its uncorrected (thin black lines) and its noise-weighted (thicker black line) forms. See text for a detailed discussion of the trends. sampled and (minimally) smoothed luminosity data above it. agree at their respective (close-to-peak) values at the center of We use the Madore-Freedman5 (hereafter MF5) edge-detector the plot where the true and/or input value resides. As is evident described in Appendix A, which samples the LF at 11 from a casual inspection of the various plots, noise-suppression optimally weighted points symmetrically placed around the results in much reduced ﬂuctuations everywhere across the output bin. Two versions of the output function are shown: the magnitude range probed by the tip detectors, without any thin solid line is the raw response-function (RRF) of the MF5 obvious degradation (or improvement) of the sought-after ﬁlter, while the thick black line is the (inversely) noise- signal at the TRGB discontinuity. We do point out, however, weighted response-function (NWRF), as described in that the width of the untreated TRGB detection is both Appendix A. The two response-functions have been scaled to asymmetric and wider than the noise-suppressed response, 6 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 4. Six subpanels illustrating the effect of increasing the photometric noise, (from 0.00 to ±0.20 mag) at ﬁxed smoothing (±0.01 mag) and small populations of RGB stars (1200). The lower portions of each of the six subpanels show the ﬁrst-derivative edge-detector output in both its uncorrected (thin black lines) and its noise- weighted (thicker black line) forms. See text for a detailed discussion of the trends. where the latter has the expected width of ±0.01 mag, which in no measurable bias in the ﬁrst-derivative response-function turn is the sampling limit of the data. The solid line marks the being used to detect the TRGB. exact position of the TRGB, and the two ﬂanking solid lines are We do, however, want to emphasize that there is no pressing again separated by ±0.01 mag for visual reference. need for smoothing the data when in this high-population, The GLOESS ﬁt to the LF faithfully tracks the discontinuity high-precision-photometry portion of parameter space; the input at 0.0 mag, in the upper panel, and the response-function, noise-weighting is sufﬁcient in suppressing spurious signals, in the lower panel, peaks precisely at the midpoint of the while simultaneously sharpening the edge-detector response. M = 0.0 mag discontinuity, in all cases. At the resolution of the In the second panel of this same ﬁgure (top right), we begin data and the detector output (0.01 mag in both cases), there is to explore the effects of adding photometric errors to the 7 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 5. Six subpanels illustrating the effect of increasing the photometric noise, (from 0.00 to ±0.20 mag) at ﬁxed smoothing (±0.01 mag) and impoverished populations of RGB stars (124). The lower portions of each of the six subpanels show the ﬁrst-derivative edge-detector output in both its uncorrected (thin black lines) and its noise-weighted (thicker black line) forms. See text for a detailed discussion of the trends. individually observed stars contributing to the simulated LF. The only effect obvious to the eye is the rounding of the All of the other parameters (in this instance, population size, originally sharp shoulders of the LF immediately above and smoothing, and the detection kernel) used in the 12 subpanels below the magnitude of the TRGB discontinuity. The dashed of Figure 2 are kept unchanged. vertical lines in the upper panel mark the 1σ smoothing radius In this second simulation, randomly generated photometric inﬂicted on the discontinuity by the degradation of the errors, having a Gaussian σ of ±0.02 mag and a mean of zero, photometry. In the subpanel below the LF, we again show have been applied randomly to each of the sampled stars, which the MF5 response-function, in both the raw (thin solid line) and were then rebinned at 0.01 mag intervals, replotted, and the noise-suppressed (solid black line) forms. Again the RRF is reanalyzed. considerably noisier overall, and it is noticeably wider (with, 8 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 6. Six subpanels illustrating the effect of increasing the photometric noise, (from 0.00 to 0.20 mag) at ﬁxed, but slightly larger smoothing (0.05 mag) than previously discussed and again for very large populations of 120,000 RGB stars. The lower portions of each of the six subpanels show the ﬁrst-derivative edge- detector output in both its uncorrected (thin black lines) and its noise-weighted (thicker black line) forms. See text for a detailed discussion of the trends. noise-induced, broad wings) at the discontinuity. The noise- error of ±0.20 mag. What is progressively different is the corrected response-function still has the bandwidth-limited decreasing signal-to-noise ratio of both response-functions as natural width of ±0.01 mag. compared to the baseline noise, at the ﬁxed baseline width of The same general trends continue as we increase the the discontinuity-sampling kernel (MF5 in this case). As the photometric errors (from ±0.05 to ±0.20 mag) in the observed slope of the LF at the TRGB discontinuity softens remaining four (lower) panels; that is, the raw response is with increased photometric errors, the power in the ﬁrst always broader than the noise-suppressed response width, derivative across a ﬁxed magnitude interval drops, while the which is stable and effectively unresolved at the ±0.01 mag Poisson population-sampling noise in the baseline LFs, on level right up to and including the largest tested photometric either side of the tip, remains largely unchanged. We 9 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 7. Six subpanels illustrating the effect of increasing the photometric noise, (from 0.00 to 0.20 mag) at ﬁxed, but slightly larger smoothing (0.05 mag) than previously discussed and for moderately large populations of RGB stars (11,331). The lower portions of each of the six subpanels show the ﬁrst-derivative edge- detector output in both its uncorrected (thin black lines) and its noise-weighted (thicker black line) forms. See text for a detailed discussion of the trends. emphasize here that the lower, response-function plots have function toward fainter magnitudes may also be a generic been sequentially rescaled for clarity, roughly normalized by feature of added noise affecting the wings. The NWRF is the peak of TRGB response. unresolved in all of the instances, regardless of the input Summary 1. For very large populations of stars deﬁning the photometric errors. As the power in the response-functions fall LF around the TRGB, the RRF, and the NWRF, each is found (with increasing photometric errors), the noise on either side to be an unbiased indicator of the position of the discontinuity and surrounding the discontinuity begin to encroach upon and in the LF marking the position of the TRGB. The RRF is found become competitive in amplitude with the declining response at to slowly but systematically increase in width with increasing the true position. This degradation is noticeable at a photometric errors. A slight skewing of the RRF distribution photometric error of ±0.10 mag, and becomes problematic 10 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 8. Six subpanels illustrating the effect of increasing the photometric noise, (from 0.00 to 0.20 mag) at ﬁxed, but slightly larger smoothing (0.05 mag) than previously discussed and for small populations of RGB stars (1240). The lower portions of each of the six subpanels show the ﬁrst-derivative edge-detector output in both its uncorrected (thin black lines) and its noise-weighted (thicker black line) forms. See text for a detailed discussion of the trends. Table 1 Guide to Simulations: Figures 2–13 and Panels (a) through (f) RGB Stars Smoothing Error at TRGB Number 0.01 0.05 0.10 0.00 0.02 0.05 0.10 0.15 0.20 120,000 Figure 2 Figure 6 Figure 10 ab c d e f 11,331 Figure 3 Figure 7 Figure 11 ab c d e f 1240 Figure 4 Figure 8 Figure 12 ab c d e f 124 Figure 5 Figure 9 Figure 13 ab c d e f 11 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. thereafter, for higher values of the photometric errors. In all population starts to contribute to an upstream ambiguity. At cases, however, the noise-suppression is effective in damping this level of smoothing, population size, and photometric error, down this background noise by about a factor of 2 compared to the tip cannot be extracted from the noise. the raw response value (see the last three panels for the worst- Summary 2. For an RGB population of approximately case examples). At the two largest values of the photometric 10,000 stars, an unambiguous detection of the tip can be errors (±0.15 and ±0.20 mag in the bottom two panels), the assured with a photometric error of ±0.05 mag or less. At a noise-induced spikes in the response-function become sufﬁ- photometric error of ±0.10 mag, the ﬁrst-detected discontinuity ciently large with respect to the declining response at the is the true one with false positives rapidly developing at fainter known and/or true position, and false-positive detections start magnitudes, downstream. However, at a photometric error of to become a problem especially downstream of the true tip. ±0.15 mag and beyond, false positives overwhelm the signal in Noise-suppression helps to damp these ﬂuctuations down, but power and in number, both below and above the true tip. does not eliminate all of the false positives in the regime of large photometric errors (>0.15 mag). 5.1.3. A Range of Photometric Errors: 1000 RGB Stars, Fixed Smoothing ±0.01 mag 5.1.2. A Range of Photometric Errors: 11,000 RGB Stars, Fixed This simulation drops the RGB population to about 1000 Smoothing 0.01 mag stars, another factor of 10 below the previous investigation. Almost immediately, at a photometric error level of At this iteration, we drop the total population of stars ±0.02 mag, the power in the response-function at the tip has contributing to the LF by about a factor of 10 (down to 11,000 dropped to a level comparable to population noise in the RGB RGB and 2000 AGB stars), keeping the smoothing at a very LF. Several false positives are seen (in the middle left panel of low level (±0.01 mag) as above, while again assessing the Figure 3) downstream of the true TRGB. Noise spikes in the effects of increased photometric errors. RGB magnitude range are so frequent (at this smoothing) that It is important to note at this point that the effects of they can randomly appear around the tip without really being decreased population size and increased photometric errors are detections of the tip. Note the cluster of noise spikes well below causally independent of each other in the plotted LFs. At ﬁxed the known position of the TRGB in the lower left panel and precision in the photometry, downsizing the population size then again a spike somewhat brighter (and certainly stronger) can only decrease the number of stars in any given bin and than the tip in the adjacent, lower right panel. thereby increase the relative error ( NN ) in that bin. The Summary 3. For a population of only 1000 RGB stars, a increased scatter in all of the panels of Figure 3 as compared to photometric error in excess of ±0.02 mag results in false Figure 2 is a direct result of the decreased number statistics and positives overwhelming the tip detection, in the absence of any can be seen repeated and progressively ampliﬁed later on in signiﬁcant smoothing (but see Section4.2 below). Figures 4 and 5 as the population size decreases further. What may not be immediately obvious is why the photometric redistribution of the data across bins at a given 5.1.4. A Range of Photometric Errors: 124 RGB Stars, Fixed population size has virtually no affect on the noise amplitude in Smoothing ±0.01 mag the LFs, seen on either side of the discontinuity. The reason for As may well have been anticipated by the trends already this is that, while this form of smoothing redistributes data seen above in the increased number of false positives as the laterally, it does not signiﬁcantly change the local mean value sample size decreased and as the photometric errors increased of N in any given bin (i.e., photometric redistribution conserves (at ﬁxed smoothing), this last simulation (shown in Figure 5) total counts within its smoothing radius). That means, of contains only 120 RGB stars, and is dominated by noise. While course, that NN is also conserved. Photometric blurring of the six-to-one contrast ratio between the RGB and the AGB individual data bin does not reduce N population noise in the population still applies, the depleted populations on either side RGB continuum; however, because of the strong asymmetry, of the jump at the TRGB are so dominated by Poisson noise inherent in the jump in the LF at the TRGB, more RGB stars that (without smoothing) both the LF itself and the tip-detection migrate to higher luminosities (and boost the apparent AGB response-function are almost indistinguishable from noise. But population) than the other way around. Accordingly, photo- with hindsight, gleaned from the upcoming panels and ﬁgures, metric errors erode the tip and systematically decrease the slope there is still (surprisingly perhaps) meaningful information on of the transition marking the rise from the AGB to RGB the position of the TRGB in all of these realizations. populations, decreasing the contrast between the AGB and the Summary 4. RGB populations of this size are insufﬁcient to tip, but still not moving the position of the discontinuity. provide reliable measurements of the tip magnitude, but some The small degree of (±0.01 mag) smoothing in these information can still be gained. simulations tracks not only the population ﬂuctuations from bin to bin but also the precisely deﬁned, sharp rise marking the 5.2. Increased Smoothing TRGB. As the photometric errors increase and the transition widens and ﬂattens the population, the power in the response- 5.2.1. A Range of Photometric Errors: 120,000 RGB Stars, Fixed function crossing the everwidening transition region starts to Smoothing ±0.05 mag drop. From a photometric error of ±0.05 mag onward (middle left panel), it is approaching the noise level of the RGB We now repeat the cycle of exploring population size effects population noise. In this simulation, there are 3–4 noise spikes and photometric errors, but now at an increased level of downstream of the true tip that are of similar power, rendering smoothing of the data set to ±0.05 mag. the identiﬁcation of the true tip ambiguous. At a photometric Once again, returning to the upper left panel of Figure 6,we error of ±0.15 mag (lower left panel), the number density of begin with an RGB population of 120,000 stars below the tip false peaks is overwhelming, and even noise in the AGB and a photometric error of 0.00 mag. At this level of precision 12 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. in the data, the discontinuity occurs between two bins, and the 5.2.4. A Range of Photometric Errors: 120 RGB Stars, Fixed smoothing is inappropriately too large, needlessly degrading Smoothing ±0.05 mag the jump. Nevertheless, the power in the ﬁrst-derivative At our smallest population size of 120 RGB stars below the response-function (bottom section of the upper right panel) is tip, the advantages of smoothing are now becoming quite very high and well deﬁned, as one might expect. And its width apparent in the ﬁrst two panels of Figure 9, illustrating the is only ±0.01 mag. Increasing the error at the tip to ±0.02 mag onset of decreased photometric precision. The ﬁrst detected tip (upper right panel) widens the discontinuity somewhat, but the is the true peak, up to an error of ±0.02 mag, after which smoothing of ±0.05 mag is still too large. The output of spurious noise peaks overwhelm the detection both in advance response-function itself responds to the increased photometric of and beyond the true position of the TRGB. Conﬁdently errors by declining in power, and widening. In the middle left detecting the true position of the TRGB in RGB populations of panel, the smoothing and the photometric errors are identical, around 100 stars in the upper magnitude range can only be and the ﬁt at the tip is almost optimal. The response-function is done with high-precision data and is still risky, given that noise well centered, continues to widen with the increased errors, and spikes of comparable power are found systematically posi- can be seen to be starting to develop structured wings that are tioned up to ±0.1 mag above and below the true tip, in virtually due to increased, but smoothed, population noise downstream all of the realizations shown here. of the TRGB. In the ﬁnal (lower right) panel, the photometric Summary 7. Increased smoothing can help compensate for errors are at their maximum for this simulation (±0.20 mag), small population sizes if the photometric quality is very good. and the response-function is widened both by the smoothing of However, using small populations is still not advisable. the discontinuity in the LF plane and by the encroaching population noise, smoothed out in the response-function plane. 5.3. Largest Smoothing Considered It is noteworthy that throughout this simulation the mode of the response-function at the true TRGB luminosity is stable at the 5.3.1. A Range of Photometric Errors: 120,000 RGB Stars, Fixed 0.01 mag level despite the widening of the response output and Smoothing ±0.10 mag the asymmetric growth of its wings. Summary 5. A larger smoothing of 0.05 mag is too large for As we now move to overly aggressive smoothing, this large data with very small photometric errors. However, this smoothing population (120,000 RGB star) simulation is clearly being becomes more appropriate when the photometric errors are oversmoothed at the tip, up to the point that the smoothing and comparable to the smoothing value. With a large population of the photometric error at the tip are of equal magnitude, RGB stars, the tip location is extremely stable in all cases. ±0.10 mag in this case. As can be seen in Figure 10,the width of the response-function at high signal-to-noise is controlled by the adopted smoothing up to the cross-over point of smoothing 5.2.2. A Range of Photometric Errors: 11,000 RGB Stars, Fixed and photometric errors (middle right panel) after which the width Smoothing ±0.05 mag grows with the increased photometric errors (last two panels).At Despite the underﬁtting of the data around the TRGB (due to high values of the photometric errors, the earlier-mentioned the larger smoothing), the ﬁlter response is sharp, unambig- wings and low-level asymmetries are still present but obviously uous, and unbiased for photometric errors less than ±0.05 mag smoothed. Again, no bias is detected in the response-function. (which coincidentally corresponds to the adopted smoothing Smoothing offers little or no advantage in the detection or here) for an RGB population size about 11,000 stars. At high measurement of the tip discontinuity in this particular scenario. photometric errors (i.e., in excess of ±0.10 mag), false Summary 8. High levels of smoothing are not advantageous positives predominate upstream (middle right panel of Figure 7) for large populations of RGB stars. but eventually crowd around and compromise the integrity of the true tip detection, encroaching both from fainter and 5.3.2. A Range of Photometric Errors: 11,000 RGB Stars, Fixed brighter magnitudes. For instance, the strongest peak in the Smoothing ±0.10 mag lower left panels is due to a smoothed version of a clustering of random noise peaks two-tenths of a magnitude below the true As in the example discussed above, dropping the RGB TRGB. The existence of a peak at the correct position in the sample to 11,000 stars in Figure 11 does not quantitatively lower right panel cannot be given much credibility given the change the description of the response-function to increased ambient noise. photometric errors. However, there is now the ﬁrst indication Summary 6. The photometric errors greater than ±0.10 mag that the mode of the response-function output is being drawn cause the false positives and potential bias in the (blended) tip off center (at the ±0.05 mag level) by the increased noise in the magnitude for populations of 11,000 RGB stars. Increased smoothed wings (last four panels). Oversmoothing should be smoothing does not mitigate this effect. carefully monitored. Running through a range of smoothing parameters can alert the user to systematic errors being introduced because of oversmoothing noise into the true peak, 5.2.3. A Range of Photometric Errors: 1200 RGB Stars, Fixed as illustrated here in the last three panels. Smoothing ±0.05 mag When numerous (comparably signiﬁcant) peaks are found Dropping the sample size by another factor of 10, down to with a low degree of smoothing, no amount of additional around 1200 RGB stars below the TRGB, does not smoothing will reveal the true peak, but rather the resulting substantially change the description of the situation as given detection will be a weighted average of the surrounding peaks, in the previous section. The detection of the tip, as seen in in which may (with enough smoothing) appear to be a single Figure 8, is relatively strong and unambiguous up to a (broad) peak; it will probably be biased: consider smoothing the photometric error of ±0.05 mag after which competing false last three panels in Figure 7, as then seen in Figure 11.Our positives begin occurring above and below the true tip. recommendation is that future investigators always try a number 13 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 9. Six subpanels illustrating the effect of increasing the photometric noise, (from 0.00 to 0.20 mag) at ﬁxed, but slightly larger smoothing (0.05 mag) than previously discussed and for impoverished populations of RGB stars 124). The lower portions of each of the six subpanels show the ﬁrst-derivative edge-detector output in both its uncorrected (thin black lines) and its noise-weighted (thicker black line) forms. See text for a detailed discussion of the trends. of smoothing kernels bracketing their preferred solution so as to Oversmoothing in this context tends to occur when the reveal the presence (or absence) of substructure that a high smoothing parameter is in excess of the photometric errors at degree of smoothing would otherwise gloss over. the TRGB. In addition, we note that a wide range of edge- Real-world investigations into selecting an optimal smooth- detection methods using different smoothing kernels (and even ing have been undertaken by Beaton et al. (2019); see their including those using maximum-likelihood ﬁtting techniques) Figures 5 and 8 for examples of the implementation of an was found to agree to very high (0.01 mag) precision when iterative smoothing analysis. There, one can see solutions that applied to the TRGB data for IC 1613 (Hatt et al. 2017). The are oversmoothed systematically drifting from their less- two papers both offer a quantitative means of selecting an smoothed solutions, being drawn away by adjacent, individu- optimal smoothing parameter, which is the one that minimizes ally low signiﬁcance, but sometimes numerous peaks. the quadrature sum of the random and systematic errors, 14 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 10. Six subpanels illustrating the effect of increasing the photometric noise, (from 0.00 to ±0.20 mag) at ﬁxed, but moderate smoothing (±0.10 mag) and for very large populations of RGB stars (120,000). The lower portions of each of the six subpanels show the ﬁrst-derivative edge-detector output in both its uncorrected (thin black lines) and its noise-weighted (thicker black line) forms. See text for a detailed discussion of the trends. generally selecting smoothing parameters that are indeed close 5.3.3. A Range of Photometric Errors: 1200 RGB Stars, Fixed Smoothing ±0.10 mag to the measured photometric errors reported for stars at the tip. Summary 9. Oversmoothing can introduce a systematic bias At 1200 stars in the RGB, tip detection is unbiased and in the presence of noise, and should be cautiously examined. unambiguous at high signal-to-noise in the photometry at the tip (upper left panel of Figure 12). At lower photometric precision, adjacent noise spikes broaden and can bias the true It should be made clear that the IC 1613 data set and its reduction are exquisite in nature given the very high precision of the photometry and the tip detection by up to 0.1 mag (bottom two panels). In this sharpness of its tip. If similar investigations were to be shown for galaxies with realization, several of the deﬂections are toward brighter lower-quality data (say due to their increased distance or mixed populations), magnitudes, but there is no reason to believe that these are they would be unlikely to demonstrate such a high level of agreement. 15 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 11. Six subpanels illustrating the effect of increasing the photometric noise, (from 0.00 to ±0.20 mag) at ﬁxed, but intermediate smoothing (±0.10 mag) of a moderately large population of RGB stars (11,000). The lower portions of each of the six subpanels show the ﬁrst-derivative edge-detector output in both its uncorrected (thin black lines) and its noise-weighted (thicker black line) forms. See text for a detailed discussion of the trends. anything more than random ﬂuctuations around the mean (see Figure 13), the true peak is properly detected, but it is not below). the highest peak over the 2 mag interval. In this particular simulation, the strongest (all false positive) peaks are found, four out of six times, at fainter magnitudes than the true tip, 5.3.4. A Range of Photometric Errors: 120 RGB Stars, Fixed and by up to 0.35 mag separation. With an average of one Smoothing ±0.10 mag star per RGB bin, wild statistical ﬂuctuations, both in the LF itself and in the discontinuity detector, are both to be This ﬁnal realization shows the ﬁlter response to a small sample size (120 RGB stars) with large (±0.10 mag) expected and are seen. This is far from being an acceptable smoothing applied to monotonically increasing photometric situation for detecting or measuring the TRGB with any errors. At high signal-to-noise (the top two panels of degree of conﬁdence. 16 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 12. Six subpanels illustrating the effect of increasing the photometric noise, (from 0.00 to ±0.20 mag) at ﬁxed, but intermediate smoothing (±0.10 mag) of a moderately large population of RGB stars (12,000). The lower portions of each of the six subpanels show the ﬁrst-derivative edge-detector output in both its uncorrected (thin black lines) and its noise-weighted (thicker black line) forms. See text for a detailed discussion of the trends. Summary 10. One should not even attempt a tip detection at function, and it alone carries little or no quantitative information low signal-to-noise in situations where the population size is on the uncertainty in the tip measurement itself. This only in the hundreds. Spurious signals will be found above and nonresponse of the width of the Sobel Function output to below the true tip. variations in smoothing and population size is shown in the three panels in Figure 14. At the bottom of each of the ﬁgures, 6. Exploring Smoothing versus Tip Uncertainty we show the Sobel-ﬁlter response to the smoothed LFs plotted above them. The two thin vertical lines centered on the In Hatt et al. (2017), and in Jang et al. (2018), it has been response-function are not determined by the the Sobel-ﬁlter shown that the output width of the Sobel response-function (and response itself, but rather they mark the input width many of its variants) is dominated by the width of the smoothing 17 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 13. Six subpanels illustrating the effect of increasing the photometric noise, (from 0.00 to ±0.20 mag) at ﬁxed, but intermediate smoothing (±0.10 mag) and an impoverished population of RGB stars (120). The lower portions of each of the six subpanels show the ﬁrst-derivative edge-detector output in both its uncorrected (thin black lines) and its noise-weighted (thicker black line) forms. See text for a detailed discussion of the trends. (±0.025 mag) of the GLOESS smoothing function, which has keeping the GLOESS smoothing the same as that in the middle nothing to do with any of the observed properties of the data. panel. The width of the response-function is unchanged, as These lines match the observed width of the Sobel-ﬁlter shown by the predicted width based on the GLOESS smoothing. response-function because the smoothing dominates. To see For a vastly different number of data points, there is no this, in the central panel, the GLOESS smoothing width has qualitative change in the width of the response-function. We end been doubled to ±0.050 mag, and the response-function is seen with where we started: the width of the Sobel-ﬁlter response- to have exactly doubled as well; same data, same population function has little or no discernible information content on the size, but twice the width of the response. The ﬁnal (right) panel uncertainty of the measured tip magnitude, and it should not be shows the effect of reducing the LF population by a factor of 10, used indiscriminately in any such applications. 18 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 14. The direct dependence of the width of the Sobel Function response to variations in smoothing (comparing the left and middle panels) and insensitivity to population size (comparing the middle and right panels). The left panel shows the Sobel-ﬁlter tip-detection response to the TRGB luminosity function, which is shown rising diagonally across the top of the plot. The two thin vertical lines mark the input width (±0.025 mag) of the GLOESS smoothing function. They match the observed width of the Sobel-ﬁlter response-function. In the central panel, the GLOESS smoothing width has been doubled to ±0.050 mag, and the response-function is seen to have doubled as well. The ﬁnal (right) panel shows the effect of reducing the luminosity function population by a factor of 10, keeping the GLOESS smoothing the same as the middle panel. The width of the response-function is unchanged, as shown by the predicted width based on the GLOESS smoothing. However, two other observables, population size and made for that study giving the uncertainty for that particular tip measurement. Failing that, these plots can be interpolated for a photometric errors, do have an expected and predictable impact ﬁrst-order estimate of the uncertainty to be associated with the on the uncertainty of the TRGB measured magnitude. This is choice of smoothing, measured population size, and known illustrated in the three panels in Figure 15 where the run of photometric errors (labeling each of the three realizations in (statistical) edge-detection errors with population size and with each of the three smoothing plots). photometric errors is shown. In each of the panels, the vertical There is one ﬁnal caveat. The simulations in this section axis gives our desired output: the standard deviation of the were undertaken for situations wherein crowding is not a major measured TRGB magnitudes. The plotted points are derived source of error at the tip. However, if observations are made in from 200 independent realizations of the LF, each smoothed regions of high surface brightness (and commensurately higher and individually scanned by our edge-detection ﬁlter. The three levels of crowding), then the simulations presented in the next line-linked symbols each track the increased precision in the section should take precedence. TRGB measurement as a function of increasing sample size for a given mean error in the TRGB photometry. The horizontal axis tracks the population size of the LF being scanned, deﬁned 7. Crowding Simulations here by the number of RGB stars in the 1 mag interval Our ﬁnal set of simulations targets the important question immediately below the TRGB. The ﬁlled circles have concerning the effect of source crowding on the RGB LF, photometric errors of ±0.10 mag. The circled plus signs especially at and around the discontinuity deﬁning the tip. have ±0.05 mag errors. And the circled dots represent error- In order to isolate and unambiguously determine the effects free photometry. The amount of GLOESS smoothing increases of crowding on the LF above the TRGB, we have set the AGB by factors of 2 (across the three panels), ranging from ±0.05 population to zero, which would ordinarily be found in the 1 to ±0.20 mag top to bottom. mag interval above the tip. We did this intentionally so as to Modulo the Poisson noise inherent in these ﬁnite simula- show what the crowded population looks like unhindered by tions, the trends are clear: All of the determinations of the superimposing it on a true AGB population; i.e., to see the uncertainty in the TRGB measured magnitude decrease signature of crowding alone. In each of Figures 16–20,we monotonically with increased sample size. And all trend lines show (in the left-hand panel) the input LF as a blue-shadowed decrease more slowly as the photometric errors increase. white line rising in number abruptly at M = −4.00 mag and Moving from ﬁgure to ﬁgure, the effects of the GLOESS thereafter exponentially increasing to form the RGB LF. The smoothing are seen to be present but subdominant. In any case, actual numbers of stars deﬁning the simulated LF are shown by for any given observation, the three parameters controlling the the black-shadowed red line, sampled at 0.01 mag intervals. In calculated uncertainty on the TRGB magnitude (the population the right panel, we show the output of the edge-detector in size, the photometric errors, and the adopted smoothing) are yellow. Running between the panels is a thin black (horizontal) known and can be input into a numerical simulation tailor- line marking the input TRGB discontinuity. In the right panel, 19 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 15. Variations of the run of edge-detection errors with population size and with photometric errors, as a function of the GLOESS smoothing changing from here through Figure 17. The vertical axis gives the standard deviation of the measured TRGB magnitudes derived from 200 independent realizations of the luminosity function for each plotted point. The three line-linked symbols each track the decrease in the TRGB uncertainty as a function of increased sample size for a given mean error in the TRGB photometry. The horizontal axis is gives the population size normalized by the number of RGB stars in the 1 mag interval immediately below the TRGB. Filled circles have photometric errors of ±0.10 mag. Circled plus signs have ±0.05 mag errors. And circled dots represent error-free photometry. The amount of GLOESS smoothing increases by factors of 2 from this to the next two ﬁgures. Here, we show the results for a ±0.05, ±0.10, and ±0.20 mag smoothing. Ripples in these trends are not signiﬁcant but due to small number statistics in the individual simulations. the position of the discontinuity as measured by the edge- 8. Eliminating AGB Stars detector is shown as a dashed black line. Finally, in the left- hand panels, in the otherwise empty space above the TRGB, we Although AGB stars above the tip are naturally found in most have inset the simulated CCD image used in the calculation halo ﬁelds, they have a fairly ﬂat LF that only serves to decrease (see the captions for the key to the various symbols marking the contrast of the tip discontinuity by placing the RGB LF on a stars in the image). slightly elevated background; but that loss of contrast does not The aforementioned simulated CCD image was instru- impose a bias, only a decrease in precision. If one considers the mental in undertaking the crowding simulation. The image TRGB discontinuity as a step function, then it is easy to visualize consists of a square array of 1000 by 1000 elements. As stars that, for reasonable high levels of the contrast ratio R > 3, say, the populated the (smooth blue line) inputLFseeninthe left measurement of the onset (when approached from brighter panel, they were randomly assigned a cell in the image, their luminosities) is not inﬂuenced by level of the baseline and/or magnitudes were converted to ﬂuxes, and they were added to background upon which it is being measured. The AGB that cell. If the cell was already occupied, the ﬂux was contribution, close to the tip, can be thought of as a relatively augmented, and the cell was considered to be crowded. The constant pedestal upon which the TRGB discontinuity is detected. process was continued until all stars were assigned to cells. The presence of true AGB stars can be eliminated by time- The summed ﬂuxes were then converted back to magnitudes, domain observations of the TRGB ﬁelds. The Gaia Mission has and these magnitudes were rebinned (at 0.01 mag resolution) shown that virtually all true AGB stars in this luminosity range thereby creating the crowded LF shownbythe jaggedred are variable (see Figure 3 in Eyer et al. 2019 for the types of line in the LF plot. variables in the Gaia CMD; and especially our Figure 8, which Summary 11. It is apparent from these simulations that self- gives the fraction of variables sitting at unity, red points seen crowding blurs the tip discontinuity. In addition, high levels of directly above the downward slanting TRGB; while no RGB crowding can cause a bias in the measured tip magnitude. In stars, at or below the tip, have been found to show variability general, it is preferred to make these measurements in low- density halo ﬁelds to avoid crowding issues. greater than 0.04 mag full amplitude; black points in the 20 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 16. A simulation of the RGB luminosity function consisting of 117,000 stars populating the magnitude range from M = −2.0, to a sharp cutoff at M = −4.0 mag deﬁning the TRGB. The individual stars are assumed to be error-free in their individual magnitudes. Noise in the luminosity function is entirely due to Poisson noise in the counts in the individual bins. Output from the Sobel-Pascal-7 edge-detector is shown in the right panel. The black horizontal line, spanning the two panels, is the TRGB magnitude set to M = −4.00 mag. The dashed black horizontal line in the right panel only is the mean value of the response-function. The inset “Simulated CCD Image” in the left panel shows the positions of all stars used in the simulation. Large yellow dots, circled in red and black, are all RGB stars from M = −4.00 to −3.90 mag. Crowded stars brighter than the TRGB are shown as larger black ﬁlled circles. This plot is given to provide a visual impression of the self-crowding of stars near the TRGB that results in the small population of stars above the tip in this particular luminosity function. aforementioned Figure 8); they would then at most contribute a side, and increase the contrast of the discontinuity deﬁning the TRGB as approached from either side. ±0.01 mag blurring of the tip. Identifying and removing the variable AGB population will aid in deblurring and decontaminating the tip from the bright 9. Unaddressed Issues: Star Formation History of the Halo on the Position and Color Dependence of the TRGB The referee has correctly pointed out that Soszynski et al. (2004) have a paper entitled “Small Amplitude Variable Red Giants in the Magellanic In a paper by Brown et al. (2003), the case has been made for Clouds.” In that paper the largest amplitude variables are above the TRGB and a signiﬁcant population of intermediate-aged (6–8 Gyr), high- consist of AGB stars alone (long-period variables, Miras, and semiregular metallicity ([Fe/H] > 0.5) RGB stars being present in the halo variables); below the tip, the amplitudes systematically decrease with period (see examples in their Figure 4), and the authors believe that these fainter of Messier 31 (M31). These stars can exceed the luminosity of variables are a mix of AGB and RGB stars. They name these stars Optical the old, metal-poor (standard) TRGB population, but they also Gravitational Lensing Experiment Small Amplitude Red Giants (OSARGs).In ascend at a color that is far to the red of the most metal-rich, old their Figure 2, the TRGB can be found at W ∼ 11.5 mag, and at TRGB stars (see their Figure 1(f)) The application of both a 1.5 < (V − I) < 2.4 in their Figure 3. In their Figure 2, cutting the lower left panel in color restricts the RGB subtip population to stars that have log blue and (especially in this case) a red cutoff to the RGB stars, P < − 1.8. Applying that cut to the period–amplitude plot in the panel directly being used to detect the tip, effectively deals with these stars. In above the period–color plot reveals that the OSARGs below the tip have peak- any case, in a forthcoming paper (Freedman et al. 2023),itis to-peak amplitudes starting at 0.04 and dropping to 0.01 mag. Converting amplitudes to equivalent σ then suggests that these very-small-amplitude shown that a comparison of TRGB distances with Cepheid variables contribute no more scatter than ±0.010–0.003 mag. The claim in the distances to the same galaxies gives a combined scatter of only subsequent literature (Anderson et al. 2023) that “every star at the TRGB is ±0.066 mag, which must bracket the total impact of all variable” is true, at the millimag level, but it is not of concern in the context of the TRGB extragalactic distance scale. remaining random errors, including the scatter that might be 21 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 17. Same as Figure 16 except this simulation involve slightly over 200,000 RGB stars in the frame. Crowding is now quite apparent in the inset CCD image, and has resulted in an appreciable population of crowded stars falling in number from the tip to 0.75 mag brighter where the self-crowding of two stars at the tip would appear as one unresolved source. For clarity, however, only one star in 10 is plotted. See text for additional details. The offset between the blue (input) line and the (red) output line in this and in the following three ﬁgures is a direct consequence of the crowding systematically producing more stars at a given magnitude by merging two or more fainter stars. introduced by star formation history difference between these than R = 15 kpc to each galaxy, and especially along the major halos. axis, which are dominated by disk stars.” Additionally, a concern about the presence of young Theory also suggests that a younger population would have populations of red giant stars in the halos of galaxies has been little impact even if it were mixed in. From model predictions of raised in the literature, most recently and extensively by McQuinn et al. (2019), it is expected that the F814W absolute McQuinn et al. (2019). Their message is mostly cautionary. We luminosity of the TRGB should have a small dependence on agree with that stance, but offer up a number of lines of stellar age of roughly 0.02 mag across an age range of 5 Gyr and evidence arguing for optimism that the effect of younger 0.04 mag across 0.5 dex. Indeed, inspection of the lower panel in populations (star formation history of the halo), if present, is a Figure 2 of McQuinn et al. shows that the effect of age spread is minor contributor to the measured scatter in the TRGB, indeed small, but it is also degenerate in its correlation with metallicity. Furthermore, this point has also been recently especially in the I band. Some of the concern about young populations interfering addressed in the single-authored paper by Hoyt (2023) where he with the detection and measurement of the TRGB is driven by states, “a long-standing question of the TRGB concerns the extent ill-fated applications of the TRGB method too close to the disk. to which age can shift the observed colors and magnitudes of The following quote from the GHOSTS Team (Monachesi TRGB stars, potentially breaking the assumption of universality in et al. 2013) summarizes the situation very well: “The CMDs any single proposed calibration (Salaris & Cassisi 2005).” are mostly populated by old RGB stars (older than 1 Gyr). Encouragingly, in this section, it was shown that the Jang & There are however younger populations such as blue, extended Lee (2017) quadratic-tip color dependence—based on observa- main-sequence (MS) stars (<500 Myr) or massive stars tions in the stellar halos of L galaxies—describes very well the burning helium in their core (25–600 Myr old red and blue TRGB magnitude–color relation in the modulation collimators loop sequence stars). These appear primarily in the ﬁelds closer (this study), Local Group dwarfs, and M33 (Rizzietal. 2007). 22 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 18. Same as Figure 16 except this simulation involves slightly over 700,000 RGB stars in the frame. Crowding is now quite apparent in the inset CCD image, and has resulted in an appreciable population of crowded stars falling in number from the tip to 0.75 mag brighter where the self-crowding of two stars at the tip would appear as one unresolved source. See text for additional details. This consistency indicates that for RGB stars found in these the cumulative holdings of TRGB distances at NASA Extra- 11 12 environments either the age distributions are identical or the age- galactic Database and Extragalactic Distance Database ). dependent variations in the I-Band TRGB magnitude are minimal. We stress that, for accuracy and precision, the method should In either case, the I-Band TRGB appears well-behaved and be applied in the outer halos of galaxies, where the effects of without a measurable bias across these host environments. The extinction and self-crowding of TRGB stars are minimal. With bottom line is as follows: if you detect blue MS stars or red these simulations, we have shown the trade-offs between a supergiant populations in your ﬁelds, then you are clearly too number of factors, including photometric precision, numbers of close to the disk, and any attempt to determine a TRGB stars deﬁning the RGB LF, and the effects of crowding. These distance to any such a line of sight through the galaxy is subject simulations can be used as a guide to optimize the choice of to systematic effects. Stay as far as possible out into the pure halo ﬁelds for accurate TRGB measurements. halo, preferably along an extension of the galaxyʼs minor axis. The above simulations presuppose that observations of the TRGB for the purpose of extragalactic distance determinations 10. Summary, Conclusions, and General Advice are being made in the halos of galaxies. Thus, they are well away from disk contamination consisting of dust, gas, and stars TRGB distances have become one of the most precise and accurate means of measuring the distances to galaxies in the of mixed ages, colors, and spatial densities. This contamination nearby universe (see, for instance, Dalcanton et al. 2009; can only degrade the TRGB detection and act (in the case of Karachentev et al. Freedman et al. 2019; Anand et al. 2022;and dust extinction) in biasing the apparent magnitude of the tip to fainter magnitudes. In terms of the lack of bias due to the AGB stars observed in these In a review of TRGB modeling, McQuinn et al. (2019) state, simulations, we note that Hatt et al. (2017) had already remarked on this “Given the building histories of halos, it is reasonable to expect (noneffect) in their own independent simulations, stating, “K we ﬁnd that the variations in ages and metallicities.” They then go on to say, AGB component simulated here has no substantial effect on the measured TRGB magnitude. The ratio of TRGB to AGB stars near the tip is about 4:1, “Assuming stellar halos are consistently metal-poor with little which might, conceivably, cause a TRGB measurement to be systematically brighter. Nonetheless, we ﬁnd that the signal-to-noise of the TRGB still https://ned.ipac.caltech.edu outweighs the noise component due to AGB stars[,] and there are minimal systematic effects.” https://edd.ifa.hawaii.edu/ 23 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 19. Same as Figure 16 except this simulation involves slightly over 1,370,000 RGB stars in the frame. Crowding is now quite apparent in the inset CCD image, and has resulted in an appreciable population of crowded stars falling in number from the tip to 0.75 mag brighter where the self-crowding of two stars at the tip would appear as one unresolved source. For clarity, only one star in 10 is plotted. See text for additional details. variation does not appear to be valid”; and they add, “To date, largely degenerate with the color dependence of the TRGB few constraints have been placed on the stellar ages.” However, luminosity on metallicity. The situation may be less straight- in their conclusions, they more optimistically state, “In the I- forward at longer wavelengths. JWST observations will be band equivalent Hubble Space Telescope F814W ﬁlter and extremely important here. JWST F090W ﬁlter, the TRGB is remarkably constant across But what do the observations of the TRGB have to say on all ages and metallicities probed.” We feel that it would be this matter? Figure 5 of Freedman et al. (2019) gives a remarkable that any halo would not have a ﬁrst generation of comparison of TRGB distances with Cepheid distances to a low-metallicity red giant stars. These stars will be the brightest variety of galaxies of different star formation histories (ages), TRGB stars in the I band and will trigger the edge-detector different mean metallicities, different distances, and different before any other higher-metallicity (potentially fainter) popula- amounts of reddening. For the entire ensemble (near and far), the combined scatter is only ±0.11 mag, which, if equally tion would enter the mix. That is to say, if there is a population of fainter, high-metallicity stars in any given halo, along with shared between the two methods, would imply that they are the generations of lower-metallicity stars that gave rise to them, each good to 4% in distance. However, if you just look at the in the marginalization process undertaken before measuring the closest sample, their σ drops to ±0.05 mag, which means that tip, the high-metallicity stars will be systematically below the the two methods are each good to 2% in distance. The ﬁrst triggering of the edge-detector and will simply augment takeaway message is that inside of that 2% all of the unresolved the RGB LF without a signature of their own (see Figure 12 in or unknown systematics are themselves contained at that same Hoyt 2023). Similar arguments can be made for the color- level, be it metallicity, age, or biased ﬁtting methods. On that rectiﬁed TRGB at longer wavelengths if the curvature down- note, we are optimistic. Having population sizes that are sufﬁcient to ﬁll the RGB ward to fainter magnitudes persists at higher metallicities (which is strongly correlated with color). Finally, it needs to be luminosity up to and including the tip is crucial to the emphasized that, as Figures 2 and 3 in McQuinn et al. (2019) extraction of an unbiased TRGB magnitude. For example, for vividly demonstrate, the color and luminosity dependence of RGB populations of less than 1000 RGB stars in the ﬁrst the TRGB on age is extremely small (in the I band), and it is magnitude interval below the true TRGB, false detections of 24 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 20. Same as Figure 16 except this simulation involves slightly over 2,000,000 RGB stars in the frame. Crowding is now quite apparent in the inset CCD image, and has resulted in an appreciable population of crowded stars falling in number from the tip to 0.75 mag brighter where the self-crowding of two stars at the tip would appear as one unresolved source. See text for additional details. the tip can be expected at the ±0.1 mag level when the of the TRGB discontinuity, and eventually bias the tip photometric precision of the data is worse than ±0.05 mag (see detection (to bright magnitudes). However, as the simulations the lower six subpanels in Figures 8, 9, 12, and 13). in Figures 16 through 20 clearly demonstrate, this effect can be Degradation of the tip due to increased photometric errors predicted by the source density of RGB stars in any given ﬁeld. can be compensated for by having increased population sizes The attempts to increase population statistics of the RGB by (compare Figures 2 and 6 with 3 and 7). moving into higher surface brightness regions of the inner halo We have demonstrated that it is best to use the least amount should be tempered because of this self-crowding effect, in of smoothing possible, commensurate with the photometric addition to line-of-sight extinction issues within the disk (that errors and population sizes. When numerous (comparably are not included in this simulation). signiﬁcant) peaks are found with a low degree of smoothing, no In the end, the characteristic (exponentially increasing) LF of amount of additional smoothing will reveal the true peak, but the faux AGB stars will betray their presence, and signal rather the resulting detection will be a weighted average of the impending bias. This could, in principle, be modeled away, but surrounding peaks, which may (with enough smoothing) might best be avoided by not observing in high-surface- appear to a be a single (broad) peak; it will probably be brightness regions to begin with. However, we do caution biased: consider smoothing the last three panels in Figure 6,as against smoothing data that are in the self-crowding regime. then seen in Figure 10. Our recommendation is that future Smoothing Figures 19 or 20 would only lead to (unnecessarily) investigators always try a number of smoothing kernels biasing the edge response to brighter magnitudes. Irreparable bracketing their preferred solution so as to reveal the presence damage to the tip detection is seen in the highest degree of self- (or absence) of substructure that a high degree of smoothing crowding simulated in Figure 20. It too could be modeled; but, would otherwise gloss over. See Figures 5 and 8 of Beaton the best solution would be to reobserve the galaxy in a region et al. (2019) for a recent implementation of this iterative of signiﬁcantly lower surface density of stars. smoothing analysis. We have simulated the CCHP adopted smoothing and Similarly, self-crowding of RGB stars near the tip results in a ﬁltering of the AGB and/or RGB LF that is being used to population of false AGB stars, which also decrease the contrast measure the magnitude of the TRGB. We ﬁnd that the width of 25 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. the Sobel-ﬁlter response-function is totally dominated by the Smoothing can take various forms. They can range from preceding GLOESS smoothing of the LF. We have, however, simple, moving rectangular averages (equally weighted) over a calibrated the run of the uncertainty in the measured value of ﬁnite number of adjacent pixels, to a more sophisticated the TRGB as a function of (a) population size and (b) smoothing using triangular, biweight and/or higher-order uncertainties in the stellar photometry at the tip. Epanechnikov weightings, all of which symmetrically decline For readers wishing to get a sense of the uncertainties on their over a ﬁnite range of pixel support and are identically zero distances in advance of making the observations, we suggest everywhere outside that range (see, for example, Silver- consulting the three panels in Figure 15. They will also be useful man 1986; also Jahne 1991). By the central limit theorem, in validating the observed error on the tip after the population multiple applications of any of these smoothing kernels size and error at the tip have been empirically deﬁned. converge on a Gaussian. A discretely sampled (digital) Finally, it should be noted that all of the recommendations Gaussian is itself, of course, a smoothing kernel (but one of being derived from these simulations (implicitly for the I band, inﬁnite support, in principle). where the run of TRGB magnitude is ﬂat with color) apply One of the earliest applied and certainly one of the most equally well to those other wavelengths once the CMDs have elementary quantized edge-detection kernels is the so-called been rectiﬁed. By rotating the data using predetermined slopes Sobel ﬁlter. This ﬁlter involves the simple differencing of pixel of the TRGB, the respective tips will also show no trend of intensity values on either side of a target position. The Sobel magnitude with color. The rectiﬁed magnitudes can then be ﬁlter, in one-dimension, takes the normalized form of [−1, 0, marginalized, and an edge-detection can be applied to the +1]. Indeed, this is the ﬁrst kernel in Figure 21 (named MF3 resulting color-corrected LFs. In support of this, recent articles, and shown in Panel (A)). At the other extreme, the ﬁrst DoG is (purely observational and mixed with modeling), both Wu et al. also a highly effective gradient detector. Invoking the binomial (2014), their Figure 5, and Durbin et al. (2020), their Figure 3, theorem once again, we recall that for very small samples show that in the near-infrared F110W (J band) there is a clearly Pascal’s triangle gives the binomial terms’ integer numbers of linear trend of TRGB luminosities with the color at least over ﬁnite-support sampling (progressively approximating, and the bluest colors ranging from 0.70 < (F110W − F160W) < eventually converging upon a Gaussian). That is well known. 0.95 mag. What is not commonly stated, but must be equally true, is that There are many additional sources of statistical and the differences between adjacent binomial terms are then systematic errors that these simulations have not explicitly discretely sampled approximations to the ﬁrst DoG. included. These uncertainties could stem from issues in Evaluating the location of the tip can then be done in assumed point-spread function (PSF) libraries, charge-transfer either of two equivalent ways: (i) ﬁnd the value of x where efﬁciency corrections, etc. While improving with time, some the output of the Sobel ﬁlter is a maximum, or (ii) ﬁnd the fraction of these issues still persists. And on top of this, there value of x where the output of the slope of Sobel ﬁlter is ﬂat. are additional systematics when dealing with PSF photometry, The Sobel ﬁlter is the ﬁrst derivative of the input function; such as errors in aperture corrections, or mismatching PSFs the latter is the second derivative of the input function—it is (due to telescope focus shifts, breathing, etc.). The list goes on, commonly known as the Laplacian. There is no difference and in light of that, our error budget should be viewed as a between the two estimations of the position of the lower limit on what is occurring in the real world. Such is the discontinuity. price paid undertaking any simulation. Pascal’s triangle of integers can also be thought of as resulting from the repeated smoothing of the initial solitary value of unit intensity by the elementary smoothing kernel [+1, 11. Epilogue +1]. ThusK0, 0, 1, 0, 0,K upon smoothing, becomes K 0, 0, Imagine we have two people approaching each other in the 1, 1, 0, 0, K and then K 0, 0, 1, 2, 1, 0, 0, K and then K 0, 0, fog. They each know that there is a cliff ahead of them, but it is 1, 3, 3, 1, 1, 0, 0, K etc. That sequence is Pascal’s triangle. too dark to see it. One is walking from the sea, approaching the Differentiating any row in Pascal’s triangle (that is, differen- cliff from below. The other is high above the water on a gently cing adjacent numbers in the triangle) is similarly visualized by sloping hill approaching the cliff from above. The ﬁrst may be having the row convolved by the zero-sum differencing kernel noticing that she is walking uphill away from the water, [+1, −1]. Applying the ﬁrst row of Pascal’s triangle gives K navigating undulating sand dunes, etc. None of this topology of 0, 0, −1, +1, 0, 0, K, which is a very compact ﬁrst derivative the local terrain can alert him to the discontinuity that he is of adjacent pixels measured at their interface. An application of walking towardK until he slams into it. The second adventurer the differencing kernel to the second line of Pascal’s triangle notices the cracks and crevasses that he has to walk over or gives K 0, 0, +1, 0, −1, 0, 0, K, which ushers in the around, but again nothing at his feet alerts him of his pending appearance of Sobel ﬁlter, as mentioned above. An application doom K until he walks off the cliff. The AGB is the sandy to the third line givesK 0, 0, −1, −1, +1, +1,K; and then the seaside below. The RGB is the grassy meadow above. Neither fourth line gives K 0, 0, −1, −2, 0, +2, +1, 0, 0, K etc. of those features can predict what lays ahead. Having said this in words, the table in Figure 22 shows the ﬁrst 13 rows of Pascal’s triangle, while the table in Figure 23 shows the ﬁrst 13 rows of the ﬁrst (digital) derivative of Pascal’s Appendix A Digital Filters and Smoothing in Edge-detection Cioni et al. (2000) were the ﬁrst to suggest the use of the Laplacian as a In most prescriptions for edge-detection in digital image means of locating the TRGB. Their approach differed somewhat from what we have discussed above, and they warn users about a potential bias between the processing, it is advised that the raw image be smoothed ﬁrst to Laplacian and the Sobel-ﬁlter solutions. We have been in communication with reduce the random noise in the image and then followed by an Dr. Cioni, and we now all agree that, when using the Laplacian, its zero additional scan of the data using a ﬁrst-derivative edge-detector crossing should be used to identify the discontinuity, and that this measure is that responds to locally detected gradients across the image. not biased with respect to the Sobel ﬁlter. 26 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 21. Binomial coefﬁcient and their ﬁrst derivatives. Each of the panels shows the values of the binomial coefﬁcients, numerically in square brackets and graphically as vertical lines in the upper half of each plot. The smooth curve is a Gaussian. In the lower half of the panel is the ﬁrst derivative of the binomial kernel, again given numerically inside of square brackets, and graphically as vertical black lines bounded by the smooth black line, which is the ﬁrst derivative of the Gaussian. triangle. In Figure 21, we show the ﬁrst four even-numbers sets Gaussian (and its derivative) each has larger support; we are of binomial coefﬁcients (MF3 through MF9) with a smooth spanning more and more pixels and thereby implicitly weight- Gaussian overlaid in the upper panels and symmetrically smoothing the data, in addition to any previous smoothing. In sampling the DoG, including its central point at the zero- Figure 24, we show the application of this single-step crossing point of the kernel, shown in the lower subpanels. methodology to a step function. Here, it is noteworthy that This immediately suggests that, in the process of going to the width of the response is largely independent of the order of higher and higher approximations, the discretely sampled the ﬁlter chosen. 27 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 22. The binomial coefﬁcients of Pascal’s triangle. Each line represents a digital kernel, each of which is a progressively higher-ﬁdelity approximation to a Gaussian. Applied to digital data, these kernels act as weighted smoothing Figure 24. Examples of the Pascal edge-detector response to a step function. functions. These are the results of applying the third, ﬁfth, and seventh (centrally peaked) ﬁlters given in Figure 22 (above), shown as red, blue, and black lines, respectively. They are scaled to equal areas, demonstrating the relatively stable width of the response-functions, independent of the smoothing width of the chosen edge-detector. Appendix B A Closing Comment about the Sensitivity of the Adopted Differencing Kernels to Structure Surrounding the Discontinuity It needs to be said in this closing remark that the discontinuity of the LF (especially as seen directly in the I band, and in the rectiﬁed LFs constructed at other wavelengths) is a very locally deﬁned quantity. By that we mean that only information contained in a handful of milli-magnitude bins, ahead of and/or following the discontinuity itself, contributes to the tipʼs detection. Moreover, the presence or absence of stars farther away from the action (i.e., from the TRGB discontinuity) can have little or no inﬂuence on the output of the edge-detection ﬁlter, since all values outside of the kernel Figure 23. First derivatives of the binomial coefﬁcients in Pascal’s triangle, as and/or ﬁlter’s support are set to zero. For instance, given the given in Figure 23 above. The ﬁrst derivative of a Gaussian (DoG) is a well- ﬁnite range of support adopted by the Sobel ﬁlter (the simplest known edge-detector in image processing. Each of the entries in this ﬁgure are example being [−1, 0, +1]), only those AGB stars that have then also edge-detectors, progressively more precise approximation to the magnitudes that are within plus or minus one bin of the TRGB DoG. Examples of their application to the detection of a step function are shown in Figure 23 (below). (AGB stars above, and RGB stars below) will have any effect 28 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. on the output of the ﬁlter at the tip. The slopes of the respective Figure 12). The two papers both offer a quantitative means of AGB or RGB LFs (be they positive, ﬂat, or negative) will only selecting an optimal smoothing parameter, which is the one that produce a constant output (of the ﬁrst-derivative ﬁlter) up until minimizes the quadrature sum of the random and systematic a signiﬁcant transition in the slope is detected (above and errors. beyond the noise). Poisson noise at the tip will serve to smooth To shed further light on that question, we offer Figure 25, the tip, but it will not bias the tip detection, nor will the ﬁlter which consists of nine subpanels containing repeated runs of care (or know) what is happening to stars more than a tenth of a the simulation displayed in subpanel (e) of Figure 11. All of the magnitude (say) above or below the tip. The presence of AGB input parameters were ﬁxed, and only the random sampling of stars immediately before the tip can only act to change the the LF was allowed to change. Details are given in the contrast in the jump by adding a pillar of counts to the extended ﬁgure caption, but our conclusions are that at this difference being measured between the AGB base (seen at one smoothing the displacements are random, but given the larger side of the differencing kernel) and (the sudden) onset of the density of false (minor) peaks below the tip as compared to RGB (seen by the other side of the advancing kernel). Nothing similar ﬂuctuations being registered above the tip, we warn that else much matters. Everything about the TRGB is local. larger amounts of smoothing will result in systematic shifts of A more compact and mathematically formal way of looking the measured tip to fainter magnitudes. at it is the fact that the derivative of a function (dF/dx) at x(i) is found in the limit as the differencing interval (dx) goes to zero at x(i) . It does not matter what F is doing at x(i + 5),or x(i + Appendix D 10),or x(i − 5),or x(i − 10), etc. Demonstration of the Lack of Bias in the Sobel Tip Detection to Smoothing of a Variety of AGB Luminosity Functions and RGB Stars above and below the TRGB Appendix C Random Displacements of the Tip Revealed in Multiple In Figures 26 through 28, we show the robust nature of the Realizations of a Single Numerical Experiment simple Sobel-ﬁlter response to the the application of smooth- ing, and to three possible forms of the AGB LF approaching At the urgings of the referee, we have investigated whether the TRGB from above. Figure 26 shows a declining AGB LF. the mismatch between the observed and true tip magnitudes is Figure 27 shows an increasing AGB LF, and Figure 28 shows a systematic or random in nature. This experiment has already ﬂat AGB LF as has been adopted in the simulations given in been run and published in the study of M101 by Beaton et al. the main paper. (2019; their Figure 5 and extended caption; and their Figure 8). As expected, given the symmetric nature of the kernels being The latter shows the effect of oversmoothing where highly applied in the smoothing and in the tip detection, there is no smoothed detections drift away from their lesser smoothed resulting bias in the position reported by the Sobel-ﬁlter versions. In addition, a wide range of edge-detection methods response-function. The effects of noise and the Sobel response using different smoothing kernels and even including those using maximum-likelihood ﬁtting techniques agreed when to smoothing is also nicely discussed in Nayar (2022), applied to the same data set for IC 1613 (Hatt et al. 2017; especially his Figures 25–27. 29 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. Figure 25. Nine subpanels illustrating a random sampling of TRGB measurements for the same number of RGB stars (1340) the same smoothing (0.10 mag) at the tip (±0.15 mag) as in subpanel (e) in Figure 11. These independently selected examples demonstrate the random drift of the peak response of the Sobel ﬁlter around the input value shown by the solid vertical black line at 0.0 mag. One peak ((b), (h), and (i)). Read left to right and top to bottom, the peak in subpanel (i) is noticeably displaced to a brighter magnitude; the example in subpanel (e) is displaced to fainter magnitudes. Other deﬂections are all within the 1σ expected deviations shown by the vertical dashed lines; ﬁve ((a), (b), (d), (g), and (h)) fall to the left, and two ((c) and (f)) fall to the right, although the latter is ﬂanked by two peaks that are apparently more signiﬁcant than the one found closest to the known answer. We note that more (low-level) structure in the output response is at magnitudes fainter than the true tip. Some of this structure is close enough (subpanels (d) and (i)) that, if excessive smoothing were applied, that action would preferentially draw the measured tip magnitudes systematically toward fainter magnitudes. 30 The Astronomical Journal, 166:2 (31pp), 2023 July Madore et al. ORCID iDs Barry F. Madore https://orcid.org/0000-0002-1576-1676 Wendy L. Freedman https://orcid.org/0000-0003- 3431-9135 Kayla A. Owens https://orcid.org/0000-0003-3339-8820 In Sung Jang https://orcid.org/0000-0002-2502-0070 References Anand, G. S., Tully, R. B., Rizzi, L., et al. 2022, ApJ, 932, 15 Anderson, R. I., Koblischke, N. W., & Eyer, L. 2023, arXiv:2303.04790 Baade, W. 1944, ApJ, 100, 137 Beaton, R. L., Seibert, M., Hatt, D., et al. 2019, ApJ, 885, 141 Berendzen, R., Hart, R., & Seeley, D. 1976, Man Discovers the Galaxies (New York: Science History Publications), 43 Brown, T. M., Ferguson, H. C., Smith, E., et al. 2003, ApJL, 592, 17 Cassisi, S., & Salaris, M. 2013, Old Stellar Populations (New York: Wiley- VCH), doi:10.1002/9783527665518 Figure 26. Toy model of a rising AGB luminosity function + an exponential Cioni, M.-R. L., van der Marel, R. P., Loup, C., et al. 2000, A&A, 359, 601 RGB luminosity function, Gaussian smoothed (upper subpanes) and convolved Conn, A. R., Lewis, G. F., Ibata, R. A., et al. 2011, ApJ, 740, 69 with a Sobel edge-detection response ﬁlter (lower subpanels). Da Costa, G. S., & Armandroff, T. E. 1990, AJ, 100, 162 Dalcanton, J. J., Williams, B. F., Seth, A. C., et al. 2009, ApJS, 183, 67 Durbin, M. J., Beaton, R. L., Dalcanton, J. J., et al. 2020, ApJ, 898, 57 Eyer, L., Rimoldini, L., Audard, M., et al. 2019, A&A, 623, 110 Freedman, W. L. 1988a, AJ, 96, 1248 Freedman, W. L. 1988b, ApJ, 326, 691 Freedman, W. L. 2021, ApJ, 919, 16 Freedman, W.L., et al. 2023, ApJ, submitted Freedman, W. L., Madore, B. F., Hatt, D., et al. 2019, ApJ, 882, 34 Frogel, J., Cohen, J., & Persson, S. E. 1983, ApJS, 53, 713 Gorski, M., Pietrzynski, G., Gieren, W., et al. 2018, AJ, 156, 278 Hatt, D., Beaton, R. l., Freedman, W. L., et al. 2017, ApJ, 845, 146 Hoyt, T. 2023, arXiv:2106.13337 Hoyt, T., Freedman, W. L., Madore, B. F., et al. 2018, ApJ, 858, 150 Jahne, B. 1991, Digital Image Processing: Concepts, Algorithms and Scientiﬁc Applications (New York: Springer), doi:10.1007/978-3-662-11565-7 Jang, I.-S., Hatt, D., Beaton, R. L., et al. 2018, ApJ, 852, 60 Jang, I.-S., & Lee, M.-G. 2017, ApJ, 835, 28 Lamers, H. G. L. M., & Levesque, E. M. 2017, Understanding Stellar Evolution (Bristol: IOP Publishing) doi:10.1088/978-0-7503-1278-3 Lee, M.G., Freedman, W.L., Madore, B.f., et al. 1993, ApJ, 417, 553 Lee, M.-G., & Jang, I.-S. 2012, ApJ, 760, 14L Madore, B. F., & Freedman, W. L. 1995, AJ, 109, 1645 Figure 27. Toy model of declining AGB luminosity + an exponential RGB Madore, B. F., Freedman, W. L., et al. 2018, ApJ, 858, 11 luminosity function, Gaussian smoothed (upper subpanes) and convolved with Madore, B. F., Mager, V., & Freedman, W. L. 2009, ApJ, 690, 389 a Sobel edge-detection response ﬁlter (lower subpanels). Madore, B. F., McAlary, C. W., McLaren, R. A., et al. 1985, ApJ, 294, 560 Makarov, D., Makarova, L., Rizzi, L., et al. 2006, AJ, 132, 2729 McQuinn, K. B. W., Boyer, M., Skillman, E. D., et al. 2019, ApJ, 880, 63 Menendez, B., Davis, M., & Moustakas, J. 2002, AJ, 124, 213 Monachesi, A., Bell, E. F., Radburn-Smith, D. J., et al. 2013, ApJ, 780, 179 Mould, J., & Kristian, J. 1986, ApJ, 305, 591 Mould, J., Kristian, J., & Da Costa, G. S. 1983, ApJ, 270, 471 Mould, J., Kristian, J., & Da Costa, G. S. 1984, ApJ, 278, 575 Nayar, S. 2022, Monograph: FPCV-2-1, Series: First Principles of Computer Vision, Computer Science (New York: Columbia Univ.), https://cave.cs. columbia.edu/Statics/monographs/Edge%20Detection%20FPCV-2-1.pdf Nikolaev, S., & Weinberg, M. D. 2000, ApJ, 542, 804 Rizzi, L., Tully, R. B., Makarov, D., et al. 2007, ApJ, 661, 815 Salaris, M., & Cassisi, S. 2005, Evolution of Stars and Stellar Populations (New York: Wiley) Sandage, A. R., & Carlson, G. 1983, ApJL, 267, 25 Scolnic, D., Riess, A.G., Wu, S., et al. 2023, arXiv:2304.06693 Serenelli, A., Weiss, A., Cassisi, S., et al. 2017, A&A, 606, 33 Shapley, H. 1918, ApJ, 48, 89 Shapley, H. 1919, ApJ, 49, 311 Shapley, H. 1930, Star Clusters (London: McGraw-Hill) Silverman, B. W. 1986, Density Estimation for Statistics and Data Analysis Figure 28. Toy model of a ﬂat AGB luminosity function + an exponential (London: Chapman and Hall) RGB luminosity function, Gaussian smoothed (upper subpanes) and convolved Soszynski, I., Udalski, A., Kubiak, M., et al. 2004, AcA, 54, 129 with a Sobel edge-detection response ﬁlter (lower subpanels). Wu, P.-F., Tully, R. B., Rizzi, L., et al. 2014, AJ, 148, 7

Journal

The Astronomical Journal – IOP Publishing

Published: Jul 1, 2023

Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Quantifying Uncertainties on the Tip of the Red Giant Branch Method

Quantifying Uncertainties on the Tip of the Red Giant Branch Method

Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Quantifying Uncertainties on the Tip of the Red Giant Branch Method

Quantifying Uncertainties on the Tip of the Red Giant Branch Method

References (42)

Abstract

Journal

Recommended Articles

There are no references for this article.

Our policy towards the use of cookies