Tracking Triggers for the HL-LHC
Ryd, Anders;Skinnari, Louise
2020-10-23 00:00:00
All rights reserved Hardware-based track reconstruction in the CMS and ATLAS trigger systems for the High-Luminosity LHC upgrade will provide unique ca- pabilities. An overview is presented of earlier track trigger systems at hadron colliders, in particular for the Tevatron CDF and D experi- ments. We discuss the plans of the CMS and ATLAS experiments to im- plement hardware-based track reconstruction for the High-Luminosity LHC. Particular focus is placed on the track trigger capability of the up- graded CMS experiment. We discuss the challenges and opportunities of this novel handle, and review the alternatives that were considered for its implementation as well as discuss the expected performance. The planned track trigger systems for CMS and ATLAS have dierent goals, and we compare and contrast the two approaches. arXiv:2010.13557v1 [physics.ins-det] 23 Oct 2020 Contents 1. INTRODUCTION ............................................................................................ 2 1.1. Challenges and Requirements at the HL-LHC .......................................................... 3 1.2. Motivation for Hardware-Based Tracking............................................................... 3 1.3. Early Developments ..................................................................................... 4 2. CMS TRACK TRIGGER ..................................................................................... 6 2.1. Outer Tracker for HL-LHC .............................................................................. 7 2.2. Trigger Architecture..................................................................................... 11 2.3. L1 Track Reconstruction Approaches................................................................... 11 2.4. HL-LHC Track-Finding System ......................................................................... 19 3. ATLAS HARDWARE-BASED TRACKING FOR HL-LHC .................................................. 21 4. SUMMARY................................................................................................... 24 1. INTRODUCTION A critical component of any high-energy particle physics collider experiment is deciding which collision events to read out and save for future analysis, and which ones to discard. At the Large Hadron Collider (LHC) (1, 2) at CERN, proton-proton (pp) collisions occur every 25 ns, corresponding to a beam crossing frequency of 40 MHz. For the ATLAS (3) and CMS (4) experiments, a typical collision event had a size of 1 MB during LHC Run 1{3. This corresponds to a data rate of roughly 40 TB per second, a rate much too large to both read out from the detectors and to store oine. Instead, only a small fraction of pp collisions is kept, made possible by the comparatively low cross sections for the physics processes of interest compared to the total LHC cross section. The system utilized to decide which collision events to retain is referred to as the trigger system. It typically consists of an initial hardware-based level implemented in custom electronics boards, as well as a high- level trigger implemented in software. The High-Luminosity LHC (HL-LHC) upgrade (5, 6), planned for installation in 2025{ 2027, will signi cantly increase the instantaneous luminosity. As a consequence, a large increase in the number of simultaneous pp interactions within the same bunch crossing, known as pileup, is also expected. This is a challenge particularly for the hardware-based trigger; novel handles are required for ATLAS and CMS to mitigate these eects. The identi cation of charged-particle trajectories (tracking) using silicon-based detec- tors is central to the LHC experiments. However, thus far, this has been restricted to software-based algorithms implemented in commercial CPUs in the high-level trigger or in the oine event reconstruction. In this review, we discuss the developments of hardware- based tracking (track triggering), speci cally utilizing inputs from silicon tracking detectors. The focus of the review is the development that is underway for the CMS experiment for the HL-LHC operation to incorporate full-detector track reconstruction in the initial trig- ger level at an input rate of 40 MHz. We discuss the motivation and physics potential for utilizing hardware-based tracking, the associated challenges at the HL-LHC, and review prior developments in this area. Details are provided about the CMS track trigger system, including the design of the new outer tracker for HL-LHC, as well as the dierent track nding approaches that have been studied, along with their implementation. We discuss the foreseen HL-LHC track- nding system that will be based on eld-programmable gate 2 A. Ryd & L. Skinnari arrays (FPGAs), the hardware platforms to be used, as well as the expected performance. The plans for the ATLAS experiment to incorporate hardware-based tracking for HL-LHC are also reviewed. We discuss the foreseen new tracking detector, two alternative trigger architectures, and the approach identi ed for performing the track reconstruction. Finally, a summary and conclusion is given. 1.1. Challenges and Requirements at the HL-LHC The HL-LHC era oers exciting physics possibilities albeit with a substantially increased instantaneous luminosity, which is experimentally challenging. The detector systems will therefore undergo signi cant upgrades in order to maximally take advantage of the physics potential of the HL-LHC data sets. The main challenge of the HL-LHC operation for the ATLAS and CMS experiments is the large increase in the number of simultaneous pp interactions. At the LHC design luminosity, an average of 25 pp interactions occur in each bunch crossing, whereas at the HL-LHC, an average of 200 simultaneous interactions are expected. The large increase in pileup results in many more particles produced in each bunch col- lision, and consequently, an ambient increased energy in the calorimeter measurements, an increased number of low-momentum muons that could be misidenti ed as high-momentum ones, and so forth. Improving the experimental handles to identify particles in the trigger is thus required. A second experimental challenge of the HL-LHC operation is high radiation levels that, in particular, the innermost detector systems must accommodate. Radiation tolerance is thus a key requirement in the design of the silicon tracker. Consequently, the ATLAS and CMS experiments will improve the trigger systems, entirely replace the in- ner silicon-based tracking detectors, and upgrade large components of the readout systems associated with the muon and calorimeter systems. 1.2. Motivation for Hardware-Based Tracking The motivation for identifying charged-particle trajectories in a hardware-based trigger system is two-fold. First, it is a powerful handle to improve the identi cation of dierent types of particles and, consequently, to enable the physics goals of the experiments. Second, the inclusion of charged particle trajectories is a novel experimental handle in the trigger that can facilitate entirely new analyses in currently unexplored corners of phase space. The HL-LHC accelerator upgrade will increase the instantaneous luminosity for pp col- 34 2 1 lisions to roughly 7:5 10 cm s , about four times larger than the maximum instanta- neous luminosity at the LHC. The target during the HL-LHC operation is to collect data sets of 3,000 fb for each of the ATLAS and CMS experiments, an order of magnitude larger than the full LHC data sets. The large data samples will enable detailed studies of rare Standard Model (SM) processes, precise Higgs boson (H ) measurements, and extend sensitivity in searches for particles and interactions beyond the SM (7, 8, 9, 10, 11). Exten- sive studies of the H boson production modes, properties, and interactions is a key goal. Rare decays, e.g. to and Z
, are expected to be conclusively observed. The H boson self-coupling, which probes the Higgs eld potential, will be studied through measuring the extremely rare HH production process. Other important tests of electroweak symmetry breaking include measurements of electroweak multiboson interactions and studies of quar- tic boson couplings, all low cross section processes that are currently not well constrained. In the
avor sector, the increased integrated luminosity will particularly bene t measure- www.annualreviews.org Tracking Triggers for the HL-LHC 3 ments of rare b and c hadron decays, an indirect probe of beyond-SM physics, and can signi cantly improve searches for e.g. top quark
avor-changing neutral currents. Direct new physics searches, e.g. for signatures of supersymmetry, dark matter candidates, or new heavy gauge bosons, will be probed to higher mass scales. The above studies rely on the capability of the experiments to identify, at the trigger level, physics objects at the electroweak scale with high eciency. Momentum thresholds must be maintained suciently low to capture leptons and hadronic decays from W=Z=H boson decays and low transverse momentum (p ) jets associated with b quarks. The searches for new physics, e.g. supersymmetry, require the identi cation of multiple low-p objects, searches for dark matter particles typically relies on the triggering capability of missing transverse momentum, and so forth. The ATLAS and CMS trigger systems for LHC Run 1{3 read out merely about 0.25% of the LHC collisions, or 100 kHz, from the initial hardware-based (Level-1, or L1) trig- ger. Following the software-based trigger, this data rate is reduced even further to about 1 kHz (12, 13). These L1 trigger systems utilize only information from the calorimeter and muon systems. To maintain the triggering capability at the HL-LHC while keeping the trig- ger rate at a manageable level, the trigger systems must be upgraded and improved. The inclusion of charged-particle tracking is a critical component in this. The precise measure- ment of a trajectory's transverse momentum improves the identi cation of muons, reducing the rate of low-p muons that are misidenti ed as a high-p ones; an example of this is T T illustrated for the CMS experiment in Fig. 1. Track information also improves electron identi cation by matching tracks to calorimeter clusters, the reconstruction of hadronic decays, and allows a selection based on tracks of charged (e; ; ) and neutral (
) particles that are isolated from other activity in the detector to reduce event rates. Beyond improving the identi cation of individual trigger objects, tracking can be used in de ning event-level quantities, e.g. the primary collision vertex to measure global quantities (e.g. the missing transverse momentum), and in correlating multiple objects when de ning trigger signatures. The inclusion of track reconstruction in the hardware-based trigger can also enable entirely new studies, probing physics processes that could previously not be identi ed in the trigger, e.g. due to overwhelming background processes when relying only on information from the calorimeter and muon system. Such signatures include e.g. displaced trajectories from hypothetical exotic long-lived particles and B physics processes with low-p nal-state particles (15). The full physics capability of incorporating tracking in the L1 trigger is yet to be explored. The LHCb experiment, specialized for studying charm and beauty physics, is being upgraded during the LHC Long Shutdown 2 in 2019{2021 (16). Since the fraction of pp collisions containing charm or beauty quarks is very high, and the instantaneous luminosity for LHCb is lower than for ATLAS and CMS, these upgrades use a dierent strategy for selecting the interesting collisions. LHCb is implementing a trigger-less system where all collisions at an input rate of about 30 MHz are read out from the detector, and the events are processed in the event lter farm, which is a fully software-based trigger. 1.3. Early Developments Track triggers for various applications have been used at many particle physics experiments. Some examples include track identi cation in e e experiments, e.g. BABAR and Belle (17, 18), and identi cation of long-lived particles, e.g. at HERA-B (19). The early applications 4 A. Ryd & L. Skinnari CMS Phase-2 Simulation 200 PU CMS Phase-2 Simulation 14 TeV Thresholds for a rate of 42 kHz (µ), 28 kHz (e) 0.8 0.6 L1 Muon P > 20 GeV Phase II KBMTF -2 0.4 L1 T L1 Global Muons rack + KBMTF Single top tt→bb!"qq 0.2 HH →bb$$ → bb!$h + " 0 20 40 60 80 100 120 140 0 20 40 60 80 100 Lepton p [GeV] gen. µ P [GeV] T T Figure 1 Example of the power of including L1 tracking in CMS for the HL-LHC. The left gure illustrates the improved muon momentum resolution when including tracks (Track + KBMTF) for the barrel muon track nder (Phase-2 KBMTF). The right gure shows simulated electron and muon p distributions from HH , single top quark, and semileptonic tt decays. The solid (dashed) vertical lines correspond to the trigger thresholds with (without) using L1 tracks for a given trigger rate, assuming 200 pileup interactions. The addition of tracking information reduces the trigger rate through the resulting improved object identi cation. From Ref. (14). of track triggers that most closely relates to the challenges at the HL-LHC were used to perform track and vertex identi cation at the CDF and D (20, 21) hadron collider experiments at the Tevatron proton-antiproton (pp ) collider at Fermilab. Developments for the ATLAS Fast TracKer (FTK) system (22) also provides important guidance toward the upgrades for the HL-LHC. Here, a brief review of the earlier track trigger developments at hadron colliders is provided. The CDF and D experiments were upgraded for Run 2 of the Tevatron to provide ad- ditional experimental capabilities. In particular, this included new or expanded capabilities for tracking and track triggering. Both experiments provided tracking in their outer track- ing detectors at the full bunch collision rate to provide improved triggering of muons and other particles. In addition, tracks in the outer trackers were combined with more precise hits from the silicon tracking detectors to provide precise impact parameter measurements for the identi cation of long-lived particles from e.g. b hadrons. The CDF and D track triggers operated in an environment with bunch crossings every 396 ns (2.5 MHz) and an average pileup of about 4 to 5. In comparison, at the HL-LHC bunch collisions will take place every 25 ns (40 MHz) with an average pileup of about 200. The CDF II Silicon Vertex Tracker (SVT) (20) made use of measurements from the Silicon Vertex Tracker (SVXII) and the Central Outer Tracker (COT) to reconstruct precise 2-D trajectories of charged particles. The CDF track trigger used two stages, the rst was the eXtremely Fast Tracker (XFT) (23, 24) that used hits from the COT to reconstruct charged particle trajectories. The XFT rst found track segments in four adjacent cells consistent with p > 1:5 GeV. The segments were linked together to form complete tracks. The SVT in the second stage was implemented using Associative Memories (AM) (25) where www.annualreviews.org Tracking Triggers for the HL-LHC 5 Efficiency Arbitrary units ! with L1 tracks # without L1 tracks e with L1 tracks e without L1 tracks the input was hits from the SVXII detector and the XFT tracks. The AM-based pattern recognition was implemented using 64 sectors in . Each sector used two AM boards that had 128 AM chips, each with 128 patterns for a total of 32k patterns for a sector. The patterns found in the AM were forwarded to the track tting stage, implemented in FPGAs. The track t was performed using a linearized t. The D experiment implemented track reconstruction in their L1 Central Track Trigger (L1CTT) (21) using hits from the Central Fiber Tracker (CFT) (26). The track nding was implemented using FPGAs in 4.5 sectors. Each sector used four FPGAs for track nding in dierent p ranges (> 10 GeV, 5 to 10 GeV, 3 to 5 GeV, and 1.5 to 3 GeV). A track was required to have hits in all eight axial CFT layers. The pattern recognition was implemented using combinatorial logic in the FPGA. The number of track equations implemented in each FPGA varied from 3,000 for the highest p range to 10,000 for the lowest p range. The tracks found in groups of ten track nding sectors were collected in an octant board, which in turn found the sectors with the highest occupancy and identi ed isolated tracks that were used for the L1 trigger decision with a latency of 2.5 s with respect to the beam collision time. The L1 tracks were also used as input to the L2 track nding. The maximum input rate for the L2 trigger was 10 kHz. The L2 processor boards were based on 1 GHz Pentium processors. The Level-2 Silicon Tracking Trigger (L2STT) used tracks found in the L1CTT and added precise Silicon Microstrip Tracker hits. This allowed rejecting misreconstructed ("fake") L1CTT tracks, improving the p resolution, and most importantly it allowed identifying tracks from long-lived particles, in particular b hadrons (27), using the precise transverse impact parameter. The ATLAS Collaboration developed a track trigger, FTK, for the Phase-1 upgrades. The FTK upgrade implemented global track reconstruction for events selected by the L1 trigger at a maximim rate of 100 kHz.. The design goal of the FTK was to reconstruct tracks with p greater than 1 GeV. The FTK used hits from the semiconductor tracker (SCT), the pixel detector, and the insertable B-layer (IBL). The hits were organized into 12 logical detector layers, eight in the SCT and four in the pixel+IBL. The FTK was implemented using AMs for the pattern recognition, using 8 of the 12 layers. In a second stage, the hits found in the pattern recognition step were used for the nal track t, including also hits matched to the track in the four layers not used in the pattern recognition. The full ATLAS FTK system was not installed, but a slice of the system was operated during the Run 2 of the LHC (28). 2. CMS TRACK TRIGGER A key goal of the CMS detector upgrades for the HL-LHC operation is to implement track nding at the L1 trigger level to keep thresholds suciently low to maintain high eciency for electroweak physics. The goal of the CMS track trigger is to reconstruct trajectories of charged particles with p > 2 GeV for all pp interactions. These tracks will be available in the L1 trigger and will provide much improved identi cation of objects such as muons, electrons, taus, and hadronic jets. To accomplish this goal, CMS has designed a silicon- based outer tracker (15) that is uniquely capable of producing trigger primitives, stubs, that are used to reconstruct the L1 tracks at 40 MHz. 6 A. Ryd & L. Skinnari 2.1. Outer Tracker for HL-LHC The CMS tracking detector will be replaced for the HL-LHC operation as the original detector would not be able to handle the expected data rate or radiation dose. The complete replacement of the tracking detector provides a unique opportunity to introduce new designs and capabilities. CMS has used this opportunity to design a tracker capable of providing p discrimination at the detector module level before readout. The CMS tracker for HL- LHC consists of an inner tracker (IT) with pixel sensors, which is not utilized in the L1 trigger, and an outer tracker (OT) with dedicated p modules. The ability to provide p T T discrimination results in a sucient reduction of the data rate to allow readout of hits above a threshold of about 2 GeV for use in the L1 trigger. This unique detector design is essential in enabling full-detector track reconstruction at the full 40 MHz bunch crossing rate. A charged particle produced at the interaction point follows a trajectory in the trans- verse plane in a uniform magnetic eld along the beam-axis given by qB = + arcsin r 1. 2p where is the azimuthal position of the trajectory at radius r, is the particle azimuthal direction at the origin, B is the magnetic eld strength, q is the particle charge, and p is the transverse momentum. For two hits with a radial separation of r, this corresponds to a separation qB = r 2. 2p where the approximation arcsin x = x has been used. The separation along the azimuthal direction x is given by qB x = rr : 3. 2p The separation of hits is proportional to r, r, B, and inversely proportional to p . The outer tracker design uses detector modules that consist of two silicon sensors sep- arated by a few millimeters. By correlating hits between the two sensors, and making use of the bending in the CMS 3.8 T magnetic eld, pairs of hits, referred to as stubs, with a small bend (high p ) can be selected. As illustrated in Fig. 2, this concept works both for barrel modules and modules in an endcap geometry. As the hit separation in the azimuthal direction is a function of the radial separation of the hits, the endcap geometry will require a larger sensor separation at large pseudorapidity (). The goal of the p modules is not to provide a precise momentum measurement, but to distinguish stubs with a transverse momentum greater than 2 GeV, a threshold that corresponds to a data volume reduction of O(10), so that these can be propagated to the back-end track nding system. The most challenging stub forming con guration is for the innermost barrel layer. At a radius of about 250 mm, a 2 GeV track will have a separation x of about 185 m for a sensor separation r of 2.6 mm. With a pitch between the silicon strips of about 100 m, p discrimination can be implemented to reject stubs from low-p trajectories. Though a larger sensor sepa- ration would increase the hit separation in the two sensors and provide a more precise p determination, the larger stub formation windows would increase the rate of combinatorial stubs. This is an issue in the inner region of the detector where the occupancy is high. In the outer part of the detector larger windows can be used since the probability of forming stubs from uncorrelated hits is smaller. www.annualreviews.org Tracking Triggers for the HL-LHC 7 Figure 2 The principle for p discrimination in the outer tracker modules. (a) Correlation of signals in closely-spaced sensors enables the rejection of low-p particles; the channels in green show the acceptance window to form a stub from the hit indicated in the inner sensor. (b) The same transverse momentum corresponds to a larger distance between the two hits at larger radii for a given sensor spacing. (c) For the endcap discs, a larger spacing between the sensors is needed to achieve the same discriminating power as in the barrel at the same radius. Figure adapted from Ref. (15). The CMS tracker will make use of two types of p modules. The rst type is the pixel-strip (PS) module in which one sensor tier consists of macro-pixels (1.446 mm long by 100 m) and the other tier of strips (2.4 cm long). The other module type is the strip-strip (2S) module where both sensor tiers are strips (5 cm long and 90 m pitch). These modules are illustrated in Fig. 3. For both PS and 2S modules, hits are read out at the edges of the module and the data are communicated between the two sensor tiers through a
ex circuit in order to correlate hits and produce stubs. Stubs are formed in the MacroPixel ASIC (MPA) (29) in the PS modules and in the CMS Binary Chip (CBC) (30) in the 2S modules. The stubs formed in the front-end ASICs are communicated to the concentrator chips (CIC) (31). There are two CICs on each module; one for each readout side. The CICs implement load balancing by grouping stubs from eight consecutive bunch crossings before sending the payload to the back-end electronics via optical links. The CIC chips implement stub p ordering and discard the stubs with lowest p in case truncation is needed. Readout data and module con guration data are transmitted and received using one Low-power Gigabit Transceiver (LpGBT) (32) per module. Depending on the location in the detector (and thus the occupancy) the optical links are operated at either 5 or 10 Gbits/s. The PS modules are used in the inner barrel layers and the inner half of the disks as they provide 3-D space points that allow precise z position measurements for the reconstructed tracks. The PS modules are also capable of handling the higher hit rate due to the ner segmentation with the macro-pixels and a smaller detector area. The detector layout is shown for one quadrant of the detector in Fig. 4. There are six barrel layers with the inner three layers composed of PS modules (TBPS) and the outer three layers composed of 2S modules (TB2S). The modules in the forward region of the three inner layers are tilted such that charged particles from the interaction point (IP) will 8 A. Ryd & L. Skinnari Figure 3 Illustration of the concept of the p modules for the upgraded CMS outer tracker for HL-LHC. The two types of modules, 2S and PS, are shown to the left and right, respectively. The top images show a layout of the two module types and the bottom images show a cross-sectional view of the connectivity at the edges of the modules. These gures illustrate how hit information is communicated between the two sensor tiers and correlations, stubs, are formed. In the 2S modules one CBC reads out the hits from both sensors and forms the correlations. For the PS modules, the strip sensor, at top in the gure, is read out by the SSA and the hits are communicated through the
exible hybrid to the MPA, which reads out the macro pixels and form the stubs. The separation between the sensors varies from 1.6 mm to 4.0 mm. From Ref. (15). traverse the modules in a direction approximately perpendicular to the sensor plane. This increases the eciency for reconstructing a stub since the particles are more likely to hit both sensors. It also reduces the sensor area needed to provide complete coverage. The PS modules in the TBPS use sensor spacings of 1.6 mm, 2.6 mm, or 4.0 mm depending on the position and orientation of the sensors, as shown in Fig. 4. The 2S modules in the barrel all have 1.8 mm spacing. There are ve disks (TEDD) on each side of the interaction point. Each disk has ve outer rings of 2S modules with sensor spacings of 1.8 mm or 4.0 mm. The two disks closest to the IP extend somewhat closer to the beamline and have ten rings of PS modules while the outer three disks have seven rings of PS modules. Also shown in Fig. 4 is the stub acceptance window, in number of strips, for which hits in the two sensors are accepted as a stub. This window varies from as little as two strips in the PS modules at the lowest radii in the forward region to nine strips. For the 2S modules the acceptance window varies between 6{15 strips. These acceptance windows are con gurable and can be tuned to manage the rate for the trigger data. The simulated stub reconstruction eciency as a function of particle p is shown in Fig. 5 for modules in the barrel and endcap regions. The stub nding windows are chosen to provide high eciency at the 2 GeV threshold for track nding. In the innermost layer, www.annualreviews.org Tracking Triggers for the HL-LHC 9 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 mm 12 10 10 9 8 1.8 11 10 9 8 7 9 8 7 7 6 2.0 800 7 7 6 6 8 6 6 2.2 9 8 6 9 9 8 7 6 7 8 7 6 5 4 8 7 7 6 6 7 7 2.4 6 6 5 6 6 6 5 6 5 2.6 1.6 mm 2.6 mm 5 4 5 4 3 5 5 4 4 4 2.8 3 4.0 mm 3 2 5 4 5 4 3 2 2 2 3.0 4.0 0 500 1000 1500 2000 2500 z [mm] Figure 4 One quarter of the layout of the CMS outer tracker for HL-LHC, showing also the dierent module spacings and stub acceptance windows used. The PS modules are indicated in light blue, yellow, and red (the PS modules in grey are not used in the trigger). The dierent colors correspond to the sensor separation in the modules; blue is 1.6 mm, yellow is 2.6 mm, and red is 4.0 mm. The 2S modules are in dark blue or red and have 1.8 mm or 4.0 mm sensor spacing, respectively. The numbers in black next to the modules are the stub acceptance windows in number of strips. The inner pixel detector modules (grey) are not used in the L1 readout. From Ref. (15). 6.3. Tracking performance 103 s=14TeV, Muons, 0 PU CMS Phase-2 Simulation CMS Phase-2 Simulation s=14TeV, Muons, 0 PU 1 1 0.8 0.8 0.6 0.6 TBPS layer 1 TEDD double-disc 1 0.4 TBPS layer 2 0.4 TEDD double-disc 2 TBPS layer 3 TEDD double-disc 3 TB2S layer 1 TEDD double-disc 4 TB2S layer 2 0.2 0.2 TEDD double-disc 5 TB2S layer 3 0 0 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10 Particle p (GeV) Particle p (GeV) T T Figure 6.5: Stub reconstruction efficiency for muons as a function of p in the barrel (left) and Figurein5the endcaps (right). Details of the efficiency turn-on and plateau are related to the choice of the stub windows. The stub nding eciency as a function of particle p for muons, shown separately for the six barrel layers (left) and for the 5 disks (right). From Ref. (15). 6.3 Tracking performance 6.3.1 Level-1 tracking performance TBPS layer 1, the turn-on curve for the stub nding eciency is less sharp than in the Two sets of tracking performance metrics were defined for the L1 tracking demonstrations (the outer layers due to the smaller bend of the track at smaller radii. L1 tracking demonstrations have been introduced in Section 3.5, and are described in great The full outer tracker design consists of 5,616 PS modules and 7,680 2S modules for a detail in Section 9.4). Fundamental tracking quantities were evaluated for a variety of physics totalscenarios, of 13,296with modules. a special Eac foc hus mo on dule tt event is read s with outhigh andpileup. controlled The demonstrations using one LpGBT also stud- module. ied tracking performance in the context of track-based algorithms that may eventually factor The o-detector readout and control is handled by 216 DTC (Data, Trigger, and Control) into the operation of the L1 trigger. This section briefly summarizes the performance studies boards. Each DTC is capable of controlling up to 72 front-end modules. The DTC receives of fundamental L1 tracking quantities. Further details on these studies are contained in the data from the front-end modules and extracts the L1 accept data from the 40 MHz trigger summaries of the individual L1 tracking demonstrations in Section 9.4. The performance of the data.“L1-oriented” The DTC unpac tracking ks the metrics trigger is discussed data and in assigns Section 12.3.1. the stubs to the correct bunch crossing; The L1 tracking hardware demonstrators, and consequently the results discussed in this sub- 10 A. Ryd & L. Skinnari section, are based on the flat barrel geometry. The performance obtained with this configura- tion is a good proxy for the performance that will be obtained with the tilted barrel. In addi- tion, based on early and later stalled R&D on through-silicon vias, the simulations assume that communication between the sensor halves in a silicon module is possible. The fundamental tracking performance metrics explored by the L1 tracking demonstrators include tracking effi- ciency, track parameter resolutions, and track rate. The three demonstrators have shown very similar performance for these metrics; for brevity, representative results from the individual approaches are given below. In the context of the demonstration effort all L1 tracking studies have been performed assuming 3 GeV stub p thresholds. Figure 6.6 presents the L1 tracking efficiency for prompt muons and electrons for tt events in a scenario with 200 pileup events on average. The tracking efficiency for muons exhibits a sharp turn-on at the 3 GeV stub p threshold, and saturates at approximately 98%. The tracking efficiency for electrons turns on more slowly and flattens out at 90%, mostly due to interaction with the detector material and consistent with the corresponding measurements of the stub r [mm] Stub efficiency Stub Efficiency it also implements the data routing for the time and space multiplexing used by the track nder boards. The optical data links from the DTCs to the track nder boards operate at 25 Gbits/s. 2.2. Trigger Architecture CMS utilizes a two-level trigger system to identify and select interesting collision events based on an initial L1 trigger, followed by a high-level trigger (HLT). The L1 trigger uses algorithms that run on custom electronics boards, while the HLT is implemented with al- gorithms that run on commercial CPUs. More recently, also algorithms running on GPUs are explored for the HLT. The L1 trigger has stringent constraints on the available process- ing time, referred to as the "latency", based on the available on-detector readout buers that must temporarily store the information from dierent collision events until a signal is received as to whether or not to further process a given event. Strict constraints are also enforced on the total accepted event rate, following the design of the subsystem readout electronics. The CMS L1 trigger system for LHC Runs 1{3 has a latency of 4 s and a maximum readout rate of 100 kHz. For the HL-LHC operation, this will be increased to a latency of 12.5 s and a maximum output rate of 750 kHz. The increased latency is necessary to accommodate the processing of L1 tracking (33). The entire CMS trigger system will be replaced for HL-LHC. At the core of the re- designed system is the added capability of L1 tracking. Figure 6 shows a schematic overview of the upgraded CMS L1 trigger architecture. Barrel and endcap calorimeter trigger sys- tems will process high-granularity information from the calorimeters. Barrel, endcap, and overlap muon track nding systems will provide triggering of muons up to jj < 2:5. The Global Track Trigger will reconstruct primary event vertices and de ne track-only based trigger objects. A two-stage correlator system will match L1 tracks with information from the calorimeter and muon systems, and perform a L1-adopted version of full event recon- struction (particle
ow reconstruction) (34), to identify physics objects such as electrons, photons, muons, hadronic leptons, jets, and energy sums. This list of objects will be propagated to the global trigger, which decides whether to retain the event for further processing (an L1 accept) or to discard it. The inclusion of tracking is critical to maintain suciently low thresholds for electroweak physics and expanding the phase space of sensitivity for beyond-SM searches. Requiring e.g. multiples jets or leptons to originate from a common collision vertex (or z position) reduces signi cantly the event rates due to accidental matches of objects from dierent overlapping pp interactions. The de nition of a complete L1 trigger menu is explored in Ref. (33), which with the inclusion of L1 tracking gives a total rate of approximately 500 kHz (keeping a 50% safety factor with respect to the maximum output rate of 750 kHz); without track information this would correspond to 4,000 kHz, a rate much too high to sustain. Incorporating tracking in the L1 trigger is thus essential in enabling the CMS HL-LHC physics program. 2.3. L1 Track Reconstruction Approaches As described in Sect. 2.1, the CMS outer tracker for HL-LHC has been designed speci cally to provide trigger primitives for L1 tracking. As part of validating the CMS tracker design, the implementation of the L1 tracking was investigated using dierent approaches, described in the Technical Design Report for the upgraded tracker (15). Three distinct approaches www.annualreviews.org Tracking Triggers for the HL-LHC 11 Figure 6 Overview of the CMS L1 trigger system for HL-LHC (14). The primary data
ow is shown with solid lines; additional data paths are under study (dashes lines). The calorimeter trigger uses Trigger Primitives (TP) from the barrel calorimeter (BC), forward calorimeter (HF), and high-granularity calorimeter (HGCAL); it is composed of a Barrel Calorimeter Trigger (BCT) and a Global Calorimeter Trigger. The muon trigger receives inputs from the Drift Tubes (DT), Resistive Plate Chambers (RPC), Cathode Strip Chambers (CSC), and Gas Electron Multiplier chambers (GEM); it has a barrel layer-1 processor and muon track nders processing data from separate regions (BMTF, OMTF, and EMTF, respectively). The Track Finders (TF) provide L1 tracks to a Global Track Trigger (GTT) as well as other components of the system. A two-stage Correlator Trigger performs correlations between tracks and other objects, including a Level-1 adopted version of particle-
ow (PF) reconstruction. The Global Trigger (GT) receives all L1 objects and issues the nal trigger decision. were studied for performing the pattern recognition: the Tracklet approach, the Hough Transform approach, and the Associative Memory (AM) approach. The rst two were fully based on FPGAs, while the latter used dedicated AM ASICs for pattern matching. Each pattern recognition approach was followed by a nal track t that improves track parameters and selects the best stub combinations in the case of combinatorics. The work required by the track t varies depending on the pattern recognition approach; approaches that make use of coarse stub positions will have additional combinatorics and fake patterns that must be rejected during the nal track t. For all approaches the track t was implemented in FPGAs. The three pursued L1 tracking approaches have in common that they split the detector into smaller geometrical regions and utilize time multiplexing to distribute the tasks of performing the pattern recognition, track tting, and duplicate removal to many dierent hardware components. Speci cally, data are organized in regions in (either in sectors spanning the full region, or in trigger towers), and organized using a round-robin 12 A. Ryd & L. Skinnari time-multiplexing system with n identical copies of the system. The choice of detector segmentation and time-multiplexing factor depends on the algorithms and are described below. The described algorithms have been implemented in rmware, and validated using hardware demonstrators. For the demonstrators developed in support of Ref. (15), the stub data pre-processing in the DTCs was not fully speci ed; the dierent approaches therefore made dierent assumptions of its functionality. This section discusses the common technology enablers necessary for L1 tracking, fol- lowed by a description of the dierent track nding approaches along with their implemen- tations. The demonstrator systems and hardware platforms validating their feasibility are also described. 2.3.1. Technology enablers. The requirements for the HL-LHC track triggers in terms of data volume and processing needs are signi cantly higher than those of earlier track trigger implementations described in Sect. 1.3. However, over the last decade there has been rapid development in several technologies that enable the implementation of a hardware track trigger for the HL-LHC. The primary challenges are the input data rate, the processing requirements to nd the tracks, and the latency required in the L1 trigger. The input data rate to the L1 track nding system is on the order of 30 Tbits/s, corresponding to about 15,000 stubs per bunch crossing every 25 ns. Sucient computational processing power is required to implement the pattern recognition and track tting algorithms. These algorithms have to process the roughly 15,000 stubs that arrive every 25 ns and nd the tracks within about 4 s. The rapid development in the last decade has provided optical links that operate at speeds of 25 Gbits/s. The use of these high speed links allows moving the data into the trigger system with a manageable number of links and with low latency. The rapid development of FPGAs provides the computational power required to implement the algorithms. The latest generation FPGAs have several thousands Digital Signal Processor (DSP) and block RAM units, and sucient routing and logic support to implement the algorithms for the track nding (https://www.xilinx.com/support/documentation/ selection-guides/ultrascale-plus-fpga-product-selection-guide.pdf, or https: //www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/pt/ intel-agilex-f-series-product-table.pdf). 2.3.2. Tracklet Approach. The tracklet approach is an implementation of a traditional road search pattern recognition algorithm. Seeds are formed from stubs in adjacent layers or disks and matching stubs are found in the other detector layers. A nal track t is performed including the stubs matched to the seed. The tracklet approach makes use of the full stub position resolution when forming the seeds. This allows nding track candidates with high purity, which reduces the resources needed in the nal track t. 2.3.2.1. Tracklet road search algorithm. Tracklets, or the seeds, are formed from pairs of stubs in adjacent layers or disks. For each stub pair a trajectory is calculated, using the beam spot as a constraint in the transverse plane, and projections to other layers and disks computed. Using the projections to other layers and disks, matching stubs are found and used in the nal track t. These steps are illustrated in Fig. 7. The seeding is performed, in parallel, in multiple combinations of layers to ensure coverage and redundancy. The main challenge in implementing this approach lies in organizing the data such that www.annualreviews.org Tracking Triggers for the HL-LHC 13 Figure 7 The tracklet approach forms seeds from pairs of stubs in adjacent layers or disks (left). The seed trajectories are projected to other layers/disks (middle) where stubs are matched and t to form the nal track (right). From Ref. (15). sucient processing capacity is available and truncation in the xed latency environment is acceptable. This is achieved by dividing the data within each sector into ner regions, referred to as virtual modules. By forming stub pairs by combining stubs in pairs of virtual modules, the number of pairs that are not consistent with a real seed can be signi cantly reduced. There are four main steps in the implementation of the tracklet approach; the stub organization, the tracklet formation, the stub matching, and the track t. The rst step organizes the stubs into regions based on the stub position and the stub z or r position in the barrel and disks, respectively. This organization of the stubs allows for the tracklet nding to proceed by rst forming candidate pairs of stubs. These stub pairs are required to be consistent with p > 2 GeV and jz j < 15 cm, and the bend of the two stubs must T 0 be consistent with the tracklet p . Stub pairs that are selected are sent to the tracklet calculator where the precise track parameters as well as projections to other layers and disks are calculated. These projections are calculated with respect to a nominal layer or disk position, and the derivatives of the and r (z) positions are also evaluated. Stub matches in the other layers and disks are rst looked for coarsely using the projection to the nominal radius; in a second stage the precise residuals are calculated using the exact stub position. The implementation selects the best stub match in each layer or disk based on the residual. 2.3.2.2. Linearized track t. For the implementation of the nal track t in the tracklet approach, a linearized t is used that takes advantage of the information from the pattern recognition step. To match stubs to the projection from the seed, the residuals between the projections and stubs are calculated in both and z (or r for disks). Using these residuals, y , and linearizing the t, we can express the nal track parameters u = ( ; ; t; z ) where is the signed radius of curvature related to q=p , is the track 0 0 T 0 azimuthal angle at the IP, t = sinh , and z is the longitudinal impact parameter as u = u + M y 4. i i where u are the track parameters from the seed and M is a weight matrix. The weight matrix can be precomputed, as to a good approximation it is independent of the parameters 14 A. Ryd & L. Skinnari except for stubs in the disks where it depends on the parameter t = sinh and the weights are tabulated for dierent ranges of t. The linear form of this equation means that the updated track parameters are obtained simply by multiply-and-add operations that are eciently implemented in hardware. Tracklet seeds with two or more matches in other layers or disks have a nal track t performed. These tracks have a minimum of four stubs. Since the track parameters from the tracklet seeds are accurately calculated based on the precise positions of the stubs, the nal track t provides a re nement to the track parameters. A linearized t that updates the track parameters of the seeds therefore works well. 2.3.2.3. Tracklet demonstrator implementation. The tracklet algorithm implementa- tion relies on extensive parallelization, both in time and space. For the hardware demon- stration of the algorithm the detector was divided into 28 sectors. Each time slice in a sector is processed by a sector processor, which is a dedicated hardware board with a modern FPGA with sucient resources for the algorithm implementation. For the demon- strator implementation, a time-multiplexing factor of six was assumed, corresponding to each sector processor receiving a new event every 150 ns. The algorithm is implemented in eight processing steps: 1) the VMRouter organizes the stubs into the virtual memories for the tracklet nding, 2) the TrackletEngine forms candidate stub pairs, 3) the TrackletCalculator calculates the precise trajectories and pro- jections, 4) the ProjectionRouter organizes the projections into virtual modules, 5) the MatchEngine forms projection stub candidate matches, 6) the MatchCalculator calculates the precise residuals between the projections and stubs, 7) the FitTrack modules perform the nal track t, and 8) the PurgeDuplicate removes duplicate tracks. Two additional pro- cessing steps were used for matching stubs to projections pointing outside the sector where the tracklet was formed. The functionality of each of these modules were implemented in rmware (Verilog) and the complete project was built from connecting up many of these modules. The calculations, primarily done in the TrackletCalculator, MatchCalculator, and FitTrack modules, are implemented using the DSP blocks in the FPGAs. The other pro- cessing modules primarily organize the data by routing data to the relevant memories that act as buers between the processing steps. The implementation is pipelined such that a dierent bunch crossing is processed at the same time in each of the eight processing steps. The tracklet algorithm was implemented using the Calorimeter Trigger Processor (CTP7) TCA board developed for the CMS Phase-1 upgrade (35). The CTP7 board has a Virtex-7 XC7VX690T FPGA and 63 input and 48 output optical links operating at speeds up to 10 Gbits/s. The demonstrator system implemented one time slice for one central sector and its two neighboring sectors. A separate board was used as the source of input stubs to the track nder boards and as the receiver of the nal tracks (36). 2.3.3. Hough Transform + Kalman Filter Approach. The Hough transform is a common tool for pattern recognition (37) and is implemented in the track nding to identify curved trajectories in the r plane. In the approach studied for Ref. (15), the hit patterns from the Hough transform are processed by a Kalman lter as the nal track t (38). 2.3.3.1. Hough transform pattern recognition. The trajectory of a charged particle produced at the origin (interaction point) that travels through a uniform magnetic eld, B, www.annualreviews.org Tracking Triggers for the HL-LHC 15 2017 JINST 12 P12019 y 1 track 5 ) 4 q/p track Figure 8 Figure 8. Illustration of the Hough Transform. On the left-hand side is a sketch of one quarter of the tracker Illustration of the Hough Transform idea for nding particle trajectories. The left-hand side shows barrel in the x-y plane, showing the trajectory of a single particle together with the stubs it produces, shown a particle trajectory with six stubs in dierent layers. The right-hand side shows the lines of possible and q=p for each of the six stubs. The point where these six lines cross corresponds as dots, in the six barrel layers. On the right-hand side, the same six stubs are now shown in Hough-space, to the track parameters for the trajectory. Adopted from Ref. (38) where the axes correspond to track parameters (q/p , ). Each stub is represented by a straight line, and T T the point where several such lines intersect both identifies a track and determines its parameters (q/p , ). T T satis es qB = r 5. 2p Particles originating at or close to the luminous region are of most relevance to the L1 trigger. where r and are the trajectory coordinates, is the angle of the trajectory at the The trajectory of such particles in the transverse plane is described by the following equation. production point, q is the particle's electric charge, and p the transverse momentum. To reduce the correlation between the r and positions it is convenient to use r = r C , where C is a radius approximately=insin the(middle ' ) of⇡ the' detector. . This gives (5.9) 2 R qB = r 6. c c Here is the angle of the track in the transverse plane at the origin [7], and the small angle 2p approximation used is valid for tracks with transverse momentum above about 2 GeV (large R). where r is a signed radial position and is the trajectory angle at the radius C . For c c each stub, at a given r and , Eq. 6 describes a straight line in the vs. q=p plane. c c T Furthermore, the (r,' ) coordinates of any stubs produced by the particle will be compatible with For a track with six stubs, as shown in Fig. 8, each of these stubs will form a unique line, this trajectory, if one neglects e ects such as multiple scattering and bremsstrahlung. where the slope is a function of the radial position of the stub. If the stubs belong to the Combining samethe particle two tra pre jectory viousthese equations, lines willone all go obtains through the same (q=p ; ) point. Tracks T c can therefore be identi ed by looking for points in the (q=p ; ) plane where multiple lines T c overlap. In an FPGA this is implemented using an array to nd bins with multiple entries. 0.0015 qB = ' · r . (5.10) This gives the track candidates with an estimate of the track parameters in the r plane. 2.3.3.2. Kalman lter track t. The Hough transform pattern recognition produces This equation shows that a single stub with coordinates (r,' ) maps onto a straight line in the track candidates that can have multiple stubs in a given layer and the nal t must there- track parameter space (q/p , ), also known as Hough-space. If several stubs are produced by fore lter the stubs T used to form the track. This ltering and tting is implemented using a Kalman lter (39). The Kalman lter begins with an initial seed estimate of the track the same particle, then the lines corresponding to these stubs in Hough-space will all intersect at parameters and their uncertainties, referred to as a Kalman state. Here, the seed is pro- a single point, neglecting e ects such as detector resolution and multiple scattering for the time vided from the Hough transform. The Kalman state is updated, iteratively, starting from stubs in the innermost layers. The seed trajectory is projected to the next layer or disk and being. This intersection of stub-lines can be used to identify track candidates. Furthermore, the the state is duplicated if there are more than one stub. The track parameters are updated coordinates of the intersection point provide a measurement of the track parameters (q/p , ). This using the position information from the next stub. The t allows for one missing layer. is illustrated in figure 8. 16 A. Ryd & L. Skinnari In this Hough-space, the gradient of each stub-line is proportional to the radius r of the stub, so is always positive. It is preferable to instead measure the radius of the stub using the variable r , defined in section 5.1. This transforms the previous equation into 0.0015 qB = ' · r , (5.11) T T where the track parameters are now (q/p , ), with defined in section 5.1. In this new Hough- T T T space, the stub-line gradient is proportional to r , so can be either positive or negative, as was assumed when drawing figure 8. The larger range of stub-line gradients improves the precision with which the intersection point can be measured, resulting in fewer misreconstructed or duplicate tracks. – 13 – This process is repeated until four stubs has been added to the trajectory. The pattern recognition (Hough transform) requires at least ve stubs, while a successful t requires four stubs. 2.3.3.3. Hough transform + Kalman lter demonstrator implementation. The Hough transform + Kalman lter approach is implemented as a Track Finding Proces- sor with four distinct components: (i) Geometric Processor (GP), responsible for the pre- processing of stub data; (ii) Hough Transform (HT), coarsely grouping stubs consistent with high-p trajectories; (iii) Kalman Filter (KF), second stage track cleaning and track tting; (iv) Duplicate Removal (DR), removing duplicate tracks. This approach has been imple- mented in rmware on FPGAs. The track nding processor handles 1=8 of the detector, an octant, and uses a time-multiplexing factor of 36. The rst step involving the Geometric Processor carries out an initial processing of the stub data as it is assumed to be received from the DTCs into an extended 64-bit format, with motivation to minimize the downstream logic requirements in the Hough Transform. It also assigns stubs to 36 sub-sectors (2 18). Each Track Finder Processor uses 36 HT arrays running in parallel, each processing stub data consistent with the corresponding geometric region de ned by the Geometric Processor; one GP sub-sector corresponds to one HT array. The HT array is split into two pipelined stages, rst the lling of the array with stubs, and second the readout of the found track candidates. Each of these two stages processed one stub at 240 MHz. The HT array is implemented in rmware as 32 columns in jq=p j and 64 rows in . The Kalman lter was implemented in two parts, a data-
ow component, which carries out the matrix operations described by the Kalman formalism (state updater and the cal- culation of track parameters and covariance matrix), as well as a control-
ow component, which manages the stub and state data. The Hough transform + Kalman lter approach was implemented using the Master Processor 7 (MP7) TCA boards developed for the CMS Phase-1 trigger upgrade (40). The MP7 board uses the same Virtex-7 FPGA as the CTP7 board. The demonstrator system used ve MP7 boards to implement the track reconstruction for one time slice for one detector octant. One MP7 board was used as the Geometrical Processor, two boards were used for the HT pattern recognition, and two boards for the KF track tter. In addition, two MP7 boards were used as data sources and one was used as a sink for the produced tracks (38). 2.3.4. Associative Memory + FPGA Approach. The associative memory approach uses a content addressable memory (CAM) ASIC to implement the pattern recognition. The AMs allow simultaneous matching to a large number of patterns, providing a fast response once all hits have been loaded. The patterns are coarse hit positions, super strips, on the modules. The AM returns all roads, i.e. patterns matched that have a minimum number of matched stubs. The nal track t implements ltering to reject wrong stub combinations. 2.3.4.1. Associative memory pattern recognition. In the AM approach the detector is divided into trigger towers. The demonstration system explored for Ref. (15) had 48 regions (8 in and 6 in ), and a time-multiplexing factor of 20. Based on the demonstration studies, about 1M patterns were determined to be required for each trigger tower and time- multiplexing slice. The design assumed that individual AM chips could be produced with www.annualreviews.org Tracking Triggers for the HL-LHC 17 about 150k patterns. The stub data received from the front-end is routed by the Data Organizers (DO) to the appropriate processing unit that is handling the stub for that BX. The stubs are converted to super strips and transmitted to the AM. The super strips vary in width from about 1 mm at the inner radii up to about 10 mm at outer radii. The larger strip size at outer radii optimizes the number of patterns required; larger strips at outer radii compensates for multiple scattering and lower occupancy. Once all stubs are loaded into the AM the pattern recognition is performed. Stubs that are on roads found by the AM are passed to the Combination Builder that forms the dierent stub combinations from stubs within each road. These are then passed onward to the track tter stage. 2.3.4.2. Track tting for the AM approach. For the AM approach, a somewhat dier- ent implementation of a linearized - t is used, which is based on a principal component analysis (PCA) technique (41). This t obtains the track parameters by multiplying a vector of stub coordinates by an appropriate translation matrix. Again, the matrix multiplication can be eciently implemented with low latency using the DSP resources in modern FP- GAs. Tabulating the transformation matrix is very simple for a set of hits in a cylindrical geometry, while in a geometry with disks or varying radial positions within a layer, the matrix depends on the track parameters, particularly . The t used has implemented a method to project the hit positions to xed radial positions. 2.3.4.3. Associative memory + FPGA demonstrator implementation. The AM+FPGA approach was implemented using AM chips that perform the stub selection and pattern recognition using coarse stub position information, followed by track tting implemented in an FPGA using stubs at full resolution. For each of the 48 trigger towers, the AM chips are preloaded with hit patterns representing possible valid trajectories for that tower's geometry. In the rst processing step, stubs are received and routed to the pattern recognition mezzanine board for the relevant trigger tower and time-multiplexing slice. The data organizer formats and loads the data into the AM. When all data has been loaded the AM pattern recognition is performed and the matched patterns are read out. The DO retrieves the stubs on the matched patterns and forward them to the track t. The demonstration of the AM+FPGA approach is based on the Pulsar2b ATCA board (42). The Pulsar2b board uses the Xilinx Virtex-7 690T FPGA and allows board-to- board communication across the ATCA backplane. The backplane communication is used in the demonstrator to implement the time multiplexing. In the demonstrator the Pulsar2b served both as the data source and the Pattern Recognition Board (PRB). The Pulsar2b supports two Pattern Recognition Mezzanine (PRM) boards that implements the pattern recognition using AMs. For the demonstrator two types of PRMs are used. One is based on the AM06 ASIC (43) and provided the pattern density required. However, as this AM ASIC was not developed for a L1 application it did not meet the latency or data throughput requirements. The second PRM was developed for AM ASICs that were under development at the time of the demonstrator. The PRM was developed such that it could support an FPGA instead of the AM ASIC. For the demonstrator, the PRM with the FPGA option is used; the FPGA provide a clock cycle accurate proxy for the AM ASIC. This allowed the demonstration to meet the latency requirement but did not have the pattern density that is ultimately required, but allowed demonstration of the rest of the system. 18 A. Ryd & L. Skinnari 1000 x24 x18 time slices, each receiving input data from 24 DTCs per ϕ nonant DTC x18 x24 -500 TFP DTC x9 -1000 processing nonants in ϕ -1000 -500 0 500 1000 x [mm] Outer Tracker Data Trigger & Control (DTC) Track Finder Processor (FTP) Figure 9 Schematic overview of the data
ow from the outer tracker, divided into nine "nonants" in , each read out by 24 DTC boards. From the DTCs, the stub data is forwarded to the track nder processor boards. 2.4. HL-LHC Track-Finding System CMS has designed a novel tracker based on the concept of p modules that will provide stub primitives at 40 MHz for the L1 tracking. The ability to reconstruct tracks in hard- ware at the L1 trigger level was demonstrated using three dierent approaches. Following these demonstrations, CMS has adopted a hybrid approach that makes use of the tracklet approach for the pattern recognition and the Kalman lter for the nal track t. This choice allows the implementation of the full algorithm in commercial FPGAs without the need to develop custom ASICs for the pattern recognition. 2.4.1. Hybrid implementation. The three described approaches for pattern recognition and track tting were pursued to demonstrate the feasibility of L1 tracking at 40 MHz in the HL- LHC environment with an average of 200 pileup interactions. The conclusion of the studies was that all three methods were feasible and could implement L1 tracking with comparable performance and meeting the latency requirement of about 4 s. For the ultimate HL-LHC system, CMS has decided to pursue an all-FPGA based approach. The primary reason for this choice is (15) to reduce the risks associated with the development of the AM ASIC, which involves new technologies such as 28 nm or 3D integration. Following the choice of an all-FPGA solution, CMS is pursuing a hybrid implementation. The tracklet algorithm is used for nding the candidate tracks, i.e. seeds are found and matched to stubs in other layers and disks, while the nal track t is performed using the Kalman lter. The combination of the tracklet approach with precise seeds and the iterative Kalman lter provides optimal performance. A schematic overview of the data ow from the outer tracker through the DTC boards and onward through the track nder processing is shown in Fig. 9. The hybrid system assumes a time-multiplexing factor of 18 and a division in into nine "nonants". 2.4.2. Hardware platforms. Developments of hardware prototypes and plans for the ul- timate system are underway to de ne the track nder boards that will be used for the HL-LHC system. The Apollo (44) platform will be used for the track nder processing www.annualreviews.org Tracking Triggers for the HL-LHC 19 y [mm] boards, while the Serenity (45) platform will be used for the DTC boards. Both platforms are based on the ATCA standard, and contain separate components for providing the nec- essary services and performing the data processing. The Apollo has a Service Module (SM) that provides infrastructure components, including the required ATCA Intelligent Platform Management Controller (IPMC) (46), powering, clock, and a system-on-module computer. A Command Module (CM) can be customized for the particular application, here the track nder processing, and contains two large FPGAs, several hundred optical ber interfaces supporting link speeds of up to 28 Gbits/s, and memories. Similarly, the Serenity platform consists of a carrier card that provides the required services, such as powering, clocking, optical interfaces, interconnections between FPGAs, IPMC functionality, and an on-board CPU for controlling the board. Daughter cards host FPGAs that are responsible for the data processing, in this case, the DTC functionality. 2.4.3. Displaced tracking. The CMS L1 track nding approaches discussed so far are de- veloped for reconstructing prompt particles. However, the unique design of the CMS Outer Tracker for HL-LHC also has the potential of reconstructing long-lived charged particles that may have escaped detection thus far. Long-lived charged particles that decay a macro- scopic distance away from the primary interaction point but prior to traversing the outer tracker would appear as displaced trajectories. With the vast physics motivation to search for such particles, see e.g. Ref. (47) for a comprehensive review, eorts are underway to explore the full potential and technical feasibility of extending the CMS L1 tracking to reconstruct displaced trajectories. Proposed initially in Ref. (48) and further explored in Ref. (49), the CMS L1 tracking could identify trajectories with displacements in the plane transverse to the beam line as large as about 10 cm. In particular, the physics case of a Higgs boson decaying to two new light scalars that in turn decay to jets, where the scalars have a suciently large lifetime so that the jets appear displaced from the primary vertex, is studied (50, 51). This process is nearly free from Standard Model background contribu- tions, and with the current CMS detector would fail to be selected in the L1 trigger, while when incorporating displaced tracking, such rare exotic decays could be probed using the large HL-LHC data set. The dedicated reconstruction of displaced trajectories could be incorporated in the tracklet pattern recognition algorithm through the addition of triplet seeds that remove the constraint to the beamspot and instead adds a third stub (48, 52). 2.4.4. Expected performance. The expected performance of the L1 track reconstruction has been studied using simulated data and validated with hardware demonstration sys- tems. The performance metrics used include the identi cation eciency of dierent types of charged particles, such as isolated muons or electrons, and charged particles from the decays of top quark{anti-quark pairs (tt), which include charged particles produced in the dense environment of jets. Other performance metrics are the expected track parameter resolutions (p , , , z ), and the total rate of tracks per event. T 0 0 Examples of the expected L1 tracking performance are illustrated in Fig. 10. It shows the expected L1 track reconstruction eciency (top left) as a function of pseudorapidity for tracks in tt events overlaid with an average of 200 pileup interactions, using the hybrid algorithm (52), as well as the expected z resolution, corresponding to intervals that en- compass either 68% or 90% of all tracks with p > 2 GeV (top right). At central , the resolution is about 1 mm, while it is less precise at higher as a consequence of the CMS outer tracker geometry with tilted PS modules. Also shown (bottom left) is the expected 20 A. Ryd & L. Skinnari track rate per sector (one ninth of the detector) for a minimum p threshold of 2 GeV. Finally, the possible extension to reconstructed displaced trajectories that do not originate from the origin, consistent with possible long-lived particles, is shown (bottom right) as the eciency as a function of the transverse impact parameter (d ) for displaced muons in events without pileup. This extended, displaced tracking has the potential to signi cantly extend the d coverage. CMS Phase-2 Simulation 14 TeV CMS Phase-2 Simulation 14 TeV 0.9 Tracks in t t +PU=200 events, p > 2 GeV 0.8 68% resolution 0.8 0.7 90% resolution 0.6 0.6 0.5 0.4 0.4 Tracks in t t +PU=200 events 0.3 p > 2 GeV 0.2 0.2 p > 8 GeV 0.1 0 0 - 2.5 - 2 - 1.5 - 1 - 0.5 0 0.5 1 1.5 2 2.5 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2 2.4 Particle η Particle |η| CMS Phase-2 Simulation 14 TeV, 200 PU - 1 CMS Phase-2 Simulation 14 TeV, PU=0 Baseline tracking - 2 Extended tracking Baseline tracking w. 5-par 0.8 Displaced muons - 3 0.6 2 < p < 20 GeV, |η| < 2.0 0.4 - 4 0.2 - 5 0 1 2 3 4 5 6 7 8 0 20 40 60 80 100 120 140 Particle d [cm] Max number of transmitted tracks per ϕ sector 0 Figure 10 L1 track nding eciency as a function of particle (top left) and track longitudinal impact parameter resolution as a function of jj (top right) for charged particles from top quark pair events with an average of 200 additional pileup interactions; L1 track rate per sector shown for a minimum p threshold of 2 GeV (bottom right); L1 track nding eciency as a function of transverse impact parameter for tracks corresponding to long-lived particles (bottom left), showing the possible enhancement in eciency from the extended, displaced tracking (14). 3. ATLAS HARDWARE-BASED TRACKING FOR HL-LHC The ATLAS experiment has designed a new tracker for the HL-LHC upgrade (53, 54). Similar to CMS, the original ATLAS tracker must be replaced as it cannot handle the much higher data rates and radiation exposure at the HL-LHC. The layout of the ATLAS tracker for the HL-LHC is shown in Fig. 11. It is based on an all-silicon design with pixel sensors in the inner regions where the occupancies are highest and strip sensors in the outer regions. The acceptance of the strip detectors covers the range jj < 2:7 while the pixels extend www.annualreviews.org Tracking Triggers for the HL-LHC 21 Tracking efficiency Fraction of events Tracking efficiency Track z resolution [cm] 0 Figure 11 The proposed geometry for the ATLAS inner tracker upgrade for the HL-LHC is an all Si based design. The innermost region, r < 350 mm, is instrumented with pixel detectors in a con guration with ve barrel layers and four rings in the forward region. The radial region 350 < r < 1000 mm is instrumented with strip detectors with four barrel layers and six forward disks. From Ref. (54). coverage to jj < 4:0. Stereo angles are implemented in the strip detectors to provide a second coordinate measurement. The ATLAS experiment's use of tracking in the trigger system diers from that of CMS described earlier in that the ATLAS tracker does not provide trigger primitives. Hence ATLAS has chosen a dierent approach to hardware-based tracking where full-detector trajectory reconstruction is not performed in the rst stage of the trigger system. The Figure 1: Top: A schematic depiction of the ITk Layout as presented in this document. Bottom: A zoomed-in view of ultimate trigger system design is under development; here we present the assumptions the pixel detector. In each case, only one quadrant and only active detector elements are shown. The active elements outlined in Ref. (55). of the strip detector are shown in blue, and those of the pixel detector are shown in red. The horizontal axis is along the beam line with zero being the interaction point. The vertical axis is the radius measured from the interaction An overview of the ATLAS trigger system is shown in Fig. 12, where both the baseline region. architecture with a single-level hardware trigger and the evolved architecture with a two- level hardware trigger, also referred to as an "L0/L1" architecture, are shown. The baseline architecture operates a single Level-0 (L0) hardware trigger using information from the calorimeter and muon detector systems, with a maximum readout rate of 1 MHz and latency of 10 s. ATLAS is additionally maintaining upgrade projects with
exibility of possibly including L1 track reconstruction through the L0/L1 architecture, where regional tracking is operating at an input rate of 4 MHz. The evolved architecture is in particular intended to mitigate risks associated with uncertainties in hadronic trigger rates and occupancies in the inner pixel detector layers (55). The primary goal of incorporating tracking in the ATLAS trigger is to reconstruct tracks in hardware as an input to the HLT to reduce CPU usage. It is foreseen to use pattern recognition with AMs (see Sect. 2.3.4.1), in combination with track tting in FPGAs, though dierent options are under consideration. In the baseline architecture, the so-called Hardware Tracking for the Trigger (HTT) is expected to perform a combination of regional 22 A. Ryd & L. Skinnari 14.3 Overview of the Evolved System Architecture Inner Tracker Calorimeters Muon System L0Calo L0Muon MUCTPI Global Trigger Event L0CTP Processor RoIE L1Track L1CTP Readout Trigger data (40 MHz) L0 accept signal Dataflow L1 accept signal Readout data (1 MHz) Event Filter Regional Readout Request ITk data (Max 4 MHz) Processor Permanent gHTT Farm Storage Readout data (800 or 600 kHz) gHTT data (100 kHz) EF accept signal Output data (10 kHz) Figure 12 (a) Baseline TDAQ System with a single-level hard- (b) Evolved TDAQ System with a two-level hardware Diagram ware trigger of the . ATLAS baseline TDAQ system (left) trigger with . a single-level hardware trigger and the evolved TDAQ system (right) with a two-level hardware trigger. From Ref. (55). Figure 14.4: Diagram of the evolved TDAQ System in Phase-II with a two-level hardware trigger (b), compared to the single-level hardware trigger configuration (a). The additional components in the evolved system are shown in light blue: the RoIE within the Global Trigger, L1Track, and L1CTP. For the ITk strip detector, the regional data will be produced by FE electronics then sent out via FELIX, while for the ITk pixel detector outer layers (Layers 2-4) and the forward rings, the regional data will be extracted from the full Level-0 data stream in FELIX, then sent out. The (seeded) and global (full-detector) tracking. Regional tracking (rHTT) would be performed remaining full detector data is sent after a L1A. Direct connections between each Level-0/Level-1 for approximately 10% of the detector with an input event rate of 1 MHz, seeded by the trigger component and the Readout system are suppressed for simplicity. L0 trigger that utilizes information from the calorimeter and muon systems only, while the global full-detector tracking (gHTT) would be performed as a second step, with an input rate of 100 kHz. The HTT system is largely based on the developments toward the FTK system that was intended to be used by ATLAS in LHC Run-3 (22). The rHTT is expected to reconstruct trajectories with p > 2 GeV, whereas the gHTT would target p > 1 GeV, T T both covering the acceptance region of jj < 4:0. Since the HTT is not operating in the rst- level trigger, the latency constraints are less stringent than for the CMS L1 track trigger system, and the allowed latency is of the order of 100 s. The HTT system is foreseen to be based on 48 independent HTT units, providing track reconstruction for a dedicated region, each corresponding to one ATCA shelf. The track nding is divided in two steps, implemented in common Tracking Processor (TP) boards. The rst stage performs the pattern recognition through AM ASICs hosted in TP boards with Pattern Recognition Mezzanines, and in the second stage, the track tting is carried out in FPGAs in TP boards with Track Fitting Mezzanines. An alternative FPGA-based, Hough Transform implementation of the pattern recogni- tion is being studied as an alternative to the use of the Associative Memory ASIC. Both the baseline AM-based as well as the FPGA-based implementations provide
exibility to enhance the selection of long-lived particles (56). www.annualreviews.org Tracking Triggers for the HL-LHC 23 4. SUMMARY One of the main challenges to fully explore the HL-LHC potential involves maintaining suf- ciently low thresholds to eciently trigger on the important electroweak physics processes in the high pileup environment. Including information from the charged-particle tracking detectors in the hardware trigger will provide a major new handle to control the trigger rates while maintaining thresholds to eciently trigger on H=W=Z boson production. The inclusion of tracking improves the quality of almost all trigger objects, including muon, electrons, hadronic s, jets, missing transverse energy, and energy sums. The CMS Collaboration has designed a new tracker for the HL-LHC upgrades where providing trigger primitives is a key novel capability. The upgraded outer tracker will be constructed using a new type of modules, p modules, which consist of two Si sensors spaced a few millimeters apart. By correlating hits in the two sensors in logic implemented on the module, trigger primitives (stubs) can be formed. These stubs, required to be consistent with charged particles with a transverse momentum above 2 GeV, can be read out at the full bunch crossing rate of 40 MHz. They serve as the input to a backend L1 track nding system. CMS has demonstrated the ability to eciently reconstruct L1 tracks in dedicated hardware within the latency requirements of the L1 trigger. Three dierent approaches were pursued for the pattern recognition plus track tting: the tracklet road search algorithm, the Hough transform plus Kalman Filter, and the Associative Memory approach. All three approaches make use of time multiplexing, where data from dierent bunch crossings are distributed to dierent processing units, and a spatial division of the detector. Each of these approaches were shown to have the potential to work and CMS is pursuing an all-FPGA based approach that uses the tracklet road search algorithm for the pattern recognition, combined with the Kalman lter for the nal t. Prototypes for the hardware for the L1 tracking, the Serenity and Apollo boards, have been developed. The Serenity board will implement the DTC functionality, which will unpack and organize the data into the correct processing boards and time multiplexing slices. The Apollo board will implement the track nding algorithm. The ATLAS Collaboration is pursuing a tracker design optimized for oine reconstruc- tion for their HL-LHC upgrades. In the baseline trigger architecture, hardware-based track reconstruction is used as an input to the software-based high-level trigger. The implementa- tion is planned to use associative memories for pattern nding and FPGA-based linearized track tting. An alternative architecture is envisaged, where low-latency regional track reconstruction is included as part of the L1 hardware trigger with an input rate of 4 MHz. Hardware-based, low latency track reconstruction will provide an important handle for the triggers at HL-LHC. The use of track information will provide an essential tool to mitigate the eects of high pileup. Signi cant work has been performed to demonstrate the feasibility of these approaches for the planned detector designs, as documented in the experiments' technical design reports for the HL-LHC upgrades. These upgrade projects are now proceeding to the implementation stage. The hardware will be installed in the experiments during LHC upgrades in 2025-2027 and operations with the upgraded detectors are scheduled to start in late 2027. 24 A. Ryd & L. Skinnari DISCLOSURE STATEMENT The authors were early proponents of the development of the CMS track trigger and the tracklet approach in particular. ACKNOWLEDGMENTS The authors thank their CMS collaborators and ATLAS friends for useful discussions and insights. A.R. is grateful to the U.S. National Science Foundation for its continued support under grant NSF-PHY-1912813. LITERATURE CITED 1. Bruning O, et al. LHC Design Report. Tech. Rep. CERN-2004-003, CERN, Geneva (2004) 2. Evans L, Bryant P. J. Instrum. 3:S08001 (2008) 3. ATLAS Collaboration. J. Instrum. 3:S08003 (2008) 4. CMS Collaboration. J. Instrum. 3:S08004 (2008) 5. Zimmermann F. PoS EPS-HEP 2009:140 (2009) 6. Apollinari G, et al. High-Luminosity Large Hadron Collider (HL-LHC): Technical Design Re- port V. 0.1. Tech. Rep. CYRM-2017-004, CERN (2017) 7. ATLAS Collaboration. Letter of Intent for the Phase-II Upgrade of the ATLAS Experiment. Tech. Rep. CERN-LHCC-2012-022. LHCC-I-023, CERN (2012) 8. ATLAS Collaboration. ATLAS Phase-II Upgrade Scoping Document. Tech. Rep. CERN-LHCC- 2015-020. LHCC-G-166, CERN (2015) 9. CMS Collaboration. Technical Proposal for the Phase-2 Upgrade of the CMS Detector. Tech. Rep. CERN-LHCC-2015-010, CERN (2015) 10. CMS Collaboration. CMS Phase II Upgrade Scope Document. Tech. Rep. CERN-LHCC-2015- 019. LHCC-G-165, CERN (2015) 11. ATLAS and CMS Collaborations. Report on the Physics at the HL-LHC and Perspectives for the HE-LHC. Tech. Rep. CYRM-2019-007, CERN (2019) 12. ATLAS Collaboration. Eur. Phys. J. C:77: 317 (2017) 13. CMS Collaboration. J. Instrum. 12:P01020 (2017) 14. CMS Collaboration. The Phase-2 Upgrade of the CMS Level-1 Trigger. Tech. Rep. CERN- LHCC-2020-004, CERN (2020) 15. CMS Collaboration. The Phase-2 Upgrade of the CMS Tracker. Tech. Rep. CERN-LHCC- 2017-009, CERN (2017) 16. LHCb Collaboration. LHCb Trigger and Online Upgrade Technical Design Report. Tech. Rep. CERN-LHCC-2014-016, LHCB-TDR-016, CERN (2014) 17. Bailey S, et al. Nucl. Instrum. Meth. A518:544 (2004) 18. Iwasaki Y, et al. IEEE Trans. Nucl. Sci. 58:1807 (2011) 19. Gerndt EKE, Xella S. Nucl. Instrum. Meth. A446:264 (2000) 20. Adelman J, et al. Nucl. Instr. Meth. A 572:361 (2007) 21. Olsen J, et al. IEEE Trans. Nucl. Sci. 51:345 (2004) 22. Shochet M, et al. Fast TracKer (FTK) Technical Design Report. Tech. Rep. CERN-LHCC- 2013-007. ATLAS-TDR-021, CERN (2013) 23. Thomson EJ, et al. IEEE Trans. Nucl. Sci. 49:1063 (2002) 24. Holm S, et al. IEEE Trans. Nucl. Sci. 47:895 (2000) 25. Dell'Orso M, Ristori L. Nucl. Instrum. Meth. A287:436 (1990) 26. Abolins M, et al. IEEE Trans. Nucl. Sci. 51:340 (2004) 27. Adams T, et al. arXiv:physics/0701195 [physics.ins-det] (2007) 28. Sottocornola S. Frascati Phys. Ser. 67:41 (2018) www.annualreviews.org Tracking Triggers for the HL-LHC 25 29. Ceresa D, et al. PoS TWEPP2018:166 (2019) 30. Prydderch ML, et al. PoS TWEPP-17:001 (2018) 31. Nodari B, et al. PoS TWEPP2018:099 (2019) 32. Mendez JM, Baron S, Kulis S, Fonseca J. PoS TWEPP2018:059 (2019) 33. CMS Collaboration. The Phase-2 Upgrade of the CMS L1 Trigger Interim Technical Design Report. Tech. Rep. CERN-LHCC-2017-013, CERN (2017) 34. CMS Collaboration. J. Instrum. 12:P10003 (2017) 35. Svetek A, et al. J. Instrum. 11:C02011 (2016) 36. Bartz E, et al. J. Instrum. 15:P06024 (2020) 37. Hough PVC. Method and means for recognizing complex patterns. Tech. Rep. U.S. patent 3,069,654 (1962) 38. Aggleton R, et al. J. Instrum. 12:P12019 (2017) 39. Billoir P, Qian S. Nucl. Instrum. Meth. A294:219 (1990) 40. Compton K, et al. J. Instrum. 7:C12024 (2012) 41. Clement E, et al. Nucl. Instrum. Meth. A935:95 (2019) 42. Ahuja, S. and others. A Full Mesh ATCA-based General Purpose Data Processing Board (Pulsar II). Tech. Rep. FERMILAB-TM-2650-E, Fermilab (2017) 43. Annovi A, et al. J. Instrum. 12:C04013 (2017) 44. Hazen ES, et al. PoS TWEPP2019:120 (2020) 45. Rose A, et al. PoS TWEPP2018:115 (2019) 46. Mendez J, et al. J. Instrum. 12:C03010 (2017) 47. Curtin D, et al. Rep. Progress Phys. 82:116201 (2019) 48. Gershtein Y. Phys. Rev. D 96:035027 (2017) 49. Gershtein Y, Knapen S. Phys. Rev. D 101:032003 (2020) 50. Cepeda M, et al. arXiv:1902.00134 [hep-ph] (2019) 51. Alimena J, et al. arXiv:1903.04497 [hep-ex] (2019) 52. James T arXiv:1910.12668 [physics.ins-det] (2019) 53. ATLAS Collaboration. Technical Design Report for the ATLAS Inner Tracker Strip Detector. Tech. Rep. CERN-LHCC-2017-005, ATLAS-TDR-025, CERN (2017) 54. ATLAS Collaboration. Expected Tracking Performance of the ATLAS Inner Tracker and the HL-LHC. Tech. Rep. ATL-PHYS-PUB-2019-014, CERN (2019) 55. ATLAS Collaboration. Technical Design Report for the Phase-II Upgrade of the ATLAS TDAQ System. Tech. Rep. CERN-LHCC-2017-020. ATLAS-TDR-029, CERN (2017) 56. M artensson M, et al. J. Instrum. 14:P11009 (2019) 26 A. Ryd & L. Skinnari
http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.pngHigh Energy Physics - ExperimentarXiv (Cornell University)http://www.deepdyve.com/lp/arxiv-cornell-university/tracking-triggers-for-the-hl-lhc-Xqrvy0dWhX