Old 2007-2014 Production Version (version 6)
Note: the Lone Word from CSC "No Data" events (always Code
word "8";
see the DMB-DDU
Format page) are processed and discarded by the DDU zero suppression
algorithm. The useful information contained in those words is summarized
in the DDU event header.
The DDU operates using a
64-bit word architecture.
The EMU DDU output data format is compatible with the CMS Common Data Format (CDF). Specifications for the CMS CDF are found here. The main CMS DAQ path uses the S-Link64 protocol and transmits 64-bit wide data, plus a "65th" bit (bit64) for use as the control character (aka "K-bit"); this bit is only true (i.e. "high") for the first header and last trailer of every event. There are additional "CMS Directives" in the CMS DAQ protocol that are specified in the CMS CDF and implemented in the DDU Format (these details below). DDU SPY data is transmitted byte-wise (from each 64-bit DDU data word) via Gigabit Ethernet lowest-byte first; the LSB (bit0) is always on the right. The SPY path data format is the same as for the S-Link path, except there is no 65th bit.
DDU Word Type | bit63<------------------>bit0
(characters are HEX placeholders) |
comment |
Header1 | 5TNN/NNNN/XXXI/IIVK | bit64 = S-Link "K-bit" set HIGH |
Header2 | 8000/0001/8000/HHHH | The "8/1/8" pattern is somewhat unique, highest bit of HHHH set to zero |
Header3 | LLLL/oooo/ZZZZ/GGMY | Highest bit of LLLL, ZZZZ & GG are set to Zero |
CSC Data | Loop through CSCs with data for the event | DMB Code word 8 is suppressed at DDU |
Trailer-2 | 8000/FFFF/8000/8000 | This "8/F/8/8" pattern is unique |
Trailer-1 | SSSS/SSSS/QQQQ/PPPP | SSSS is detailed DDU status/error at End of Event, highest bit of QQQQ is DMB Full flag, highest bit of PPPP set to Zero |
Trailer | A?WW/WWWW/RRRR/UUMK | bit64 = S-Link "K-bit" set HIGH |
T = Event Type (4 bits)
N = Event Number (24 bits)
X = BX number (12bits)
I = Source ID (10+2 bits) //set to decimal 760 (2F8h) for TF S-Link; otherwise it is "dRR" where RR is the RUI
V = Format Version (4 bits) //this is DDU Format Version 6
K = S-Link Status (lowest 4 bits are reserved in each S-Link64 K-word)
H = DMBs that reached Full FIFO (15 bits)
L = DDU Inputs Connected to "Live" CSCs (15 bits)
o = DDU Output Path Status (16 bits)
Z = DDU Inputs (i.e. CSCs) with Data for this Event (15 bits)
G = DDU Beginning of Event Status (7 bits)
M = DDU TTS/FMM Status (4 bits)
Y = Count of CSCs with Data for this Event (4 bits)
S = DDU Status/Error (32 bits)
Q = CSCs in Error State (15 bits)
P = CSCs in Warning State (15 bits)
W = DDU 64-bit Word Count (24 bits)DDU_WC = (6 + 25*N_ts*nCFEB + 3*nDMB) + 64*nTMB + 99*nALCTR = CRC Check Word (16 bits)
U = Event Status (8 bits) {repeat some elements of DDU Status here}
? = not used yet, but reserved for future use (4 bits each)
More details below
bit[63:60] = 0x5 //hard-coded in DDU firmware by CMS directive
bit[59:55] = 4-bit "event type" //CMS directive, no details yet
bit[55:32] = 24-bit Level 1 event number //from DDU L1A counter
bit[31:20] = 12-bit Bunch Crossing Number //by CMS directive*may come from DDU, DMB, ALCT or TMBbit[19:8] = 12-bit Source ID //CMS directive, no details yet
*appropriate matching req'd at DDU
bit[7:4] = 4-bit "format version" //set to "6" for 2007-2014 Production DDU, 5 for previous version, 3-4 for preproduction boards
bit[3:0] = 4-bits reserved //CMS directive, no details yet*special "S-Link64 Protocol" status bits
bit[63:16] = a 48-bit constant, hard-coded in DDU firmware: 0x8000/0001/8000*this is a "somewhat unique DDU word" near the start of the event packet; rare, but not guaranteed to be uniquebit[15] = 0, hard-coded in DDU firmware
bit[14:0] = shows which DMBs have reached a FIFO Full state (High True)
bit[63] = 0, hard-coded in DDU firmware
bit[62:48] = 15-bits indicating which DDU Fiber Inputs have a "Live" fiber (High True, 1 Fiber Input per CSC)
bit[47:32] = 16-bits indicating the DDU Output Path status, defined below:bit47: DDU Output-Limited Buffer Overflow occurred (may be OK)bit[31] = 0, hard-coded in DDU firmware*an output path (GbE/SPY or DCC/S-Link) could not handle the rate, resulting in a buffer overflowbit46: DAQ Wait was asserted by S-Link or DCC (status only)
bit45: Link Full (LFF) was asserted by S-Link (status only)
bit44: DDU S-Link Never Ready (status only)
bit43: GbE/SPY FIFO Overflow occurred (status only)
bit42: GbE/SPY Event was skipped to prevent overflow (status only)
bit41: GbE/SPY FIFO Always Empty (status only)
bit40: Gbe/SPY Fiber Connection Error occurred (status only)
bit39: DDU Buffer Overflow caused by DAQ Wait (RESET req'd)*S-Link could not handle the rate, resulting in a buffer overflowbit38: DAQ Wait is set by DCC/S-Link (status only)
bit37: Link Full (LFF) is set by DDU S-Link (status only)
bit36: Not Ready is set by DDU S-Link (status only)
bit35: GbE/SPY FIFO is Full (status only)
bit34: GbE/SPY Path was Not Enabled for this event (status only)*no SPY data was sent due to prescale or throughput limitationbit33: GbE/SPY FIFO is Not Empty (status only)
bit32: DCC Link is Not Ready (may be OK)
bit[30:16] = 15-bits indicating which CSCs have data for this event; one bit allocated per DDU fiber input
bit15 = 0, hard-coded in DDU firmware
bit[14:8] = 7-bit DDU Beginning of Event Status, defined below:bit14: DDU single event warning (possible data problem, may be OK; RESET?)bit[7:4] = 4-bit DDU TTS/FMM Status //current TTS status as reported to FMM, see CDF specification*minor format error, fiber/RX error, or the DDU lost it's clock for some time; possible data lossbit13: DDU SyncError (bad event, RESET req'd)
*consider RESET if this warning continues for consecutive events*Multiple L1A errors or FIFO Full; possible data lossbit12: DDU detected Fiber Error (hardware configuration change, RESET req'd)*change of fiber connection status or No Live Fibers; a hardware problem probably existsbit11: DDU detected Critical Error, irrecoverable (RESET req'd)*OR of all possible "RESET required" casesbit10: DDU detected Single Error (bad event)*OR of all possible "bad" cases at Beginning of Eventbit9: DDU detected DMB L1A Match Error (bad event, RESET?)*the DDU L1A event number match failed for 1 or more CSCs; possible one-time bit errorbit8: DDU Timeout Error (bad event, RESET req'd)
*if error continues for consecutive events then RESET req'd*data from a CSC never arrived
*an unknowable amount of data has been irrevocably lost
bit[3:0] = number of CSCs which have data for this event
bit[63:0] = a 64-bit constant, hard-coded in DDU firmware: 0x8000/FFFF/8000/8000*this is a "DDU unique word" near the end of the event packet
bit[63:32] = 32-bit DDU status/error code, defined below:   (32-bit word, shifted by 32 bits)bit63: CSC LCT/DAV Mismatch occurred (bad event)bit[31] = DMB Full Flag (status only)
bit62: DDU-CFEB L1 Number Mismatch occurred (bad event)
bit61: No Good DMB CRCs were detected in this Event (perfectly normal empty event or possible bad event?)
bit60: CFEB Count Error occurred (bad event)
bit59: DDU Bad First Data Word From CSC Error (bad event)
bit58: DDU L1A-FIFO Full Error (RESET req'd)*the DDU L1A-event info FIFO went full; some triggers/events may be lost or garbledbit57: DDU Data Stuck in FIFO Error (RESET req'd)
bit56: DDU NoLiveFibers Error (status only)*no DDU fiber inputs are connected, something is wrong; will cause other errors...bit55: DDU Special Word Inconsistency Warning (possible bad event?)*a bit-vote failure occured on an input fiber channelbit54: DDU Input FPGA Error (bad event)
bit53: DCC/S-Link Wait is set (status only)
bit52: DCC Link is Not Ready (may be OK)
bit51: DDU detected TMB Error (bad event)*TMB trail word not found or TMB L1A, CRC or wordcount inconsistentbit50: DDU detected ALCT Error (bad event)*ALCT trail word not found or ALCT L1A, CRC or wordcount inconsistentbit49: DDU detected TMB or ALCT Word Count Error (bad event, RESET?)*TMB/ALCT wordcount inconsistentbit48: DDU detected TMB or ALCT L1A Number Error (bad event, RESET?)
*if error continues for consecutive events then RESET req'd*TMB/ALCT L1A Number mismatch with DDUbit47: DDU detected Critical Error, irrecoverable (RESET req'd)
*if error continues for consecutive events then RESET req'd*OR of all possible "RESET required" casesbit46: DDU detected Single Error (bad event)*OR of all possible "bad event" casesbit45: DDU Single Warning (possible bad event?)*OR of bit55, bit42bit44: DDU FIFO Near Full Warning or DAQ Wait is set (status only)*OR of all possible "Near Full" casesbit43: DDU detected Data Alignment Error from 1 or more inputs (bad event)*CSC data violated the 64-bit word boundarybit42: DDU Clock-DLL Error (may be OK, RESET?)*the DDU lost it's clock for an unknown period of time; some triggers/events/data may be lostbit41: DDU detected CSC Error (bad event)*Timeout, DMB CRC, CFEB Sync/Overflow, or missing CFEB databit40: DDU Lost In Event Error (bad event, but end was found)*the DDU failed to find an expected control word within the event, NOT fatalbit39: DDU Lost In Data Error (bad event, RESET req'd)*usally Fatal; DDU checking algorithms are irrevocably lost in the data streambit38: DDU Timeout Error (bad event, RESET req'd)
*mis-sequenced data structure, possible that different events were run together
*found at least one of the following in the event data stream, all of which are very bad:-Extra CSC_First_Word before CSC_Last_Word
-Extra DMB_Header2 before DMB_Last_Word
-Lone Word before DMB_Last_Word
-Extra TMB/ALCT_Trailer before DMB_Last_Word
-Extra DMB_Trailer1 before DMB_Last_Word
-DMB_Trailer2 before DMB_Trailer1Note: CSC_Last_Word == DMB_Trailer2*data from a fiber input either never started or never finishedbit37: DDU detected TMB or ALCT CRC Error (bad event, RESET?)
*an unknowable amount of data has been irrevocably lost*CRC check failed on 1 or more TMB/ALCT; possible one-time bit errorbit36: DDU Multiple Transmit Errors (bad event, RESET req'd)
*if error continues for consecutive events then RESET req'd*one bit-vote failure (or Rx Error) has occured on multiple occassions for the same CSCbit35: DDU Sync Lost/Buffer Overflow Error (bad event, RESET req'd)*an unknowable amount of data has been irrevocably lostbit34: DDU detected Fiber Error (hardware configuration change, RESET req'd)*change of connection status on 1 or more DDU fiber inputs; a hardware problem probably existsbit33: DDU detected DMB or CFEB L1A Match Error (bad event, RESET?)*the DDU L1A event number match failed for 1 or more CSC boards; possible one-time bit errorbit32: DDU detected DMB or CFEB CRC Error (bad event, RESET?)
*if error continues for consecutive events then RESET req'd*CRC check failed for ADC data on 1 or more CFEBs; possible one-time bit errorNote: error bits requiring RESET persist until the RESET occurs.
*if error continues for consecutive events then RESET req'd
bit[30:16] = shows which CSCs are in an Error state
bit[15] = 0, hard-coded in DDU firmware
bit[14:0] = shows which CSCs are in a Warning state
Trail (S-Link64 DAQ Control word)bit[63:60] = 0xA //hard-coded in DDU firmware by CMS directive
bit[59:56] = 4-bits unspecified //CMS directive, not assigned yet so I used them...*currently use bit58 as "Track Finder data detected" flag, and bit59 as "Track Finder data error" flag*currently use bit56 as "Close L1As" flag that indicates events closer than 1000 nsecbit[55:32] = 24-bit DDU Word Count for the Event //number of 64-bit words sent from DDU
bit[31:16] = 16-bit Event CRC
bit[15:8] = 8-bit Event Status //from DDU; lower 4 bits by CMS directive, higher 4 bits are not assigned so I used them...*currently equivalent to Trail-1 bits [47:40]bit[7:4] = 4-bit DDU TTS/FMM Status //current TTS status as reported to FMM, see CDF specification
bit[3:0] = 4-bits reserved //used for CMS CDF protocol
Link to second DDU Prototype (small green DDU) Output Data Format.Link to old DDU Prototype (big red DDU) Output Data Format.
- Useful for beamtest data from October 2004.
- Also good for big red DDU at CERN Slice test (old DDU with updated firmware).
- Useful for beamtest data from May 2003 through June 2004.
- Also good for big red DDU at UCLA (has original firmware).
Revision Summary:
- 23 October 2002: Original version (J. Gilmore, gilmore@mps.ohio-state.edu)
- 5 December 2002: Updated status decode in Header3 (bit57, 40, 39) and added comments in table (J. Gilmore)
- 1 May 2003: Modify trigger board info now that TMB and ALCT have separate data flows (J. Gilmore)
- 19 April 2004: Fix error in Header1 "fov" description (J. Gilmore)
- 4 May 2004: Fix typo in bit50 (J. Gilmore)
- 24 Sept. 2004: Modify for New DDU; code 9's are kept now, last Trailer data is rearranged to match CMS directive, DDU Status bit37 redefined (J. Gilmore)
- 11 July 2005: Modify for Production DDU; redefined Headers 2&3 and Trail-1 (J. Gilmore)
- 24 Aug 2005: Update for production history (J. Gilmore)
- 28 April 2006: Update definition for SourceID and Beginning of Event Status (J. Gilmore)
- 13 October 2006: Improved trailer status definitions (J. Gilmore)
- 16 November 2006: Added links to CMS C.D.F. specification (J. Gilmore)
- 30 November 2007: New ALCT/TMB Format version, FOV=6 now (J. Gilmore)