CMS EMU Prototype DDU:  Output Data Format

Second Prototype (and First-Updated) Version

link to Production DDU Format

DDU Transmission Format

The DDU will send at least some minimal information for EVERY L1A received, even if there was no useful CSC data to read out.  This minimal DDU data consists of 3 HEADER words (header 1, 2, 3) and 3 TRAILER words (trail-2, trail-1, trail).  Any CSC data for the event (if present) is contained between header3 and trail-2. The CSC data is read out sequentially from every DMB source that has data for the event.

Note:  the Lone Word from DMB "No Data" events (always Code word   "8"; see the DMB-DDU Format page) are processed and discarded by the DDU zero suppression algorithm.  The useful information contained in those words is summarized in the DDU event header. The DDU operates using a 64-bit word architecture.
 

The DDU output format is equivalent in both the Main (S-Link64) and Private EMU (Gigabit Ethernet) DAQ paths:

  • The main CMS DAQ path follows the S-Link64 protocol and transmits 64-bit wide data, plus a "65th" bit (bit64) for use as the control character (aka "K-bit"); this bit is only true (i.e. "high") for the first header and last trailer of every event.  There are other "CMS Directives" for the S-Link DAQ protocol that are implemented in the DDU Format; these details are specified below.
  • DDU data is transmitted byte-wise via Gigabit Ethernet lowest-byte first; the LSB (bit0) is always on the right.

    • Overview of the DDU event record

    Critical event information is encoded in the format of the DDU headers and trailers, as shown below.
    DDU Word Type bit63<------------------>bit0
    (characters are HEX placeholders)    		 comment
    Header1 5TNN/NNNN/XXXI/IIVK bit64 = S-Link "K-bit" set HIGH
    Header2 8000/0001/8000/8000 This "8/1/8/8" pattern is unique
    Header3 SSSS/SSSS/ZZZZ/000Y Y tells how many DMBs have data, ZZZZ tells which DMBs they are, SSSS is detailed DDU status/error at BOE
    CSC Data Loop through DMBs with data for the event DMB Code word 8 is suppressed at DDU
    Trailer-2 8000/FFFF/8000/8000 This "8/F/8/8" pattern is unique
    Trailer-1 SSSS/SSSS/ZZZZ/000Y like Header3 with updated SSSS during event processing
    Trailer A?WW/WWWW/RRRR/UUMK bit64 = S-Link "K-bit" set HIGH
    Where:
    T = Event Type (4 bits)
    N = Event Number (24 bits)
    X = BX number (12bits)
    I = Source ID (10+2 bits)
    V = Format Version (4 bits)
    K = S-Link Status (lowest 4 bits are reserved in each S-Link64 K-word)
    S = DDU Status/Error (32 bits)
    Z = DMB-DAV (16 bits)
    Y = Active DMB Count (4 bits)
    W = DDU 64-bit Word Count (24 bits)
    DDU_WC = (6 + 25*N_ts*nCFEB + 3*nDMB)
    R = CRC Check Word (16 bits)
    U = Event Status (8 bits) {repeat some elements of DDU Status here}
    M = DDU TTS Status (4 bits)
    ? = not used yet, but reserved for future use (4 bits each)
    More details below

    Header1 (S-Link64 DAQ Control word)
    bit[63:60] = 0x5       //hard-coded in DDU firmware by CMS directive
    bit[59:55] = 4-bit "event type"       //CMS directive, no details yet
    bit[55:32] = 24-bit Level 1 event number       //from DDU L1A counter
    bit[31:20] = 12-bit Bunch Crossing Number       //by CMS directive
    *may come from DDU, DMB, ALCT or TMB
    *appropriate matching req'd at DDU
    bit[19:8] = 12-bit Source ID          //CMS directive, no details yet
    bit[7:4] = 4-bit "format version"   //CMS directive, no details yet
    bit[3:0] = 4-bits reserved               //CMS directive, no details yet
    *special "S-Link64 Protocol" status bits


    Header2

    bit[63:0] = a 64-bit constant, hard-coded in DDU firmware:     0x8000/0001/8000/8000
    *this is a "DDU unique word" near the start of the event packet


    Header3

    bit[63:32] = 32-bit DDU status/error code for EMU use, defined below:
    bit63: DDU Output Limited Buffer Overflow (status only)
    bit62: DDU Giga-Bit Ethernet FIFO Full Warning (status only)
    bit61: DDU Giga-Bit Ethernet FIFO Near Full Warning (status only)
    bit60: DDU Giga-Bit Ethernet Fiber Error (status only!)
    bit59: DDU Bad First Data Word From DMB Error (bad event)
    bit58: DDU L1A-FIFO Full Error (RESET req'd)
    *the DDU L1A-event info FIFO went full; some triggers/events may be lost or garbled
    bit57: DDU Data Stuck in FIFO Error (RESET req'd)
    bit56: DDU NoLiveFibers Error (status only)
    *no DDU fiber inputs are connected, something is wrong; will cause other errors...
    bit55: DDU Special Word Inconsistency Warning (possible bad event?)
    *a bit-vote failure occured on an input fiber channel
    bit54: DDU Ethernet FPGA clock-DLL Error (status only!)
    *the Ethernet system on the DDU lost it's clock for an unknown period of time
    *only the private Ethernet readout path is affected by this
    bit53: DDU S-Link Full Bit set (status only)
    bit52: DDU S-Link Not Ready (status only)
    bit51: DDU detected TMB Error (bad event)
    *TMB trail word not found or TMB L1A, CRC or wordcount inconsistent
    bit50: DDU detected ALCT Error (bad event)
    *ALCT trail word not found or ALCT L1A, CRC or wordcount inconsistent
    bit49: DDU detected TMB or ALCT Word Count Error (bad event, RESET?)
    *TMB/ALCT wordcount inconsistent
    *if error continues for consecutive events then RESET req'd
    bit48: DDU detected TMB or ALCT L1A Number Error (bad event, RESET?)
    *TMB/ALCT L1A Number mismatch with DDU
    *if error continues for consecutive events then RESET req'd
    bit47: DDU detected Critical Error, irrecoverable (RESET req'd)
    *OR of all possible "RESET required" cases
    bit46: DDU detected Single Error (bad event)
    *OR of all possible "bad event" cases
    bit45: DDU Single Warning (possible bad event?)
    *OR of bit55, bit42
    bit44: DDU FIFO Near Full Warning (status only)
    *OR of all possible "Near Full" cases
    bit43: DDU detected RX Error on 1 or more input fibers (bad event, RESET req'd)
    *future DDU designs will handle this differently...
    bit42: DDU Control FPGA clock-DLL Error (lost phase lock, RESET req'd)
    *the DDU lost it's clock for an unknown period of time; some triggers/events/data may be lost
    bit41: DDU detected SCA Full Error (B-code received via DMB, some timesamples lost, bad event)
    bit40: DDU Lost In Event Error (bad event, but end was found)
    *the DDU failed to find an expected control word within the event, NOT fatal
    bit39: DDU Lost In Data Error (bad event, RESET req'd)
    *usally Fatal; DDU checking algorithms are irrevocably lost in the data stream
    *mis-sequenced data structure, possible that different events were run together
    *found at least one of the following in the event data stream, all of which are very bad:
    -Extra DMB_Header1 before DMB_Last_Word
    -Extra DMB_Header2 before DMB_Last_Word
    -Lone Word before DMB_Last_Word
    -Extra TMB/ALCT_Trailer before DMB_Last_Word
    -Extra DMB_Trailer1 before DMB_Last_Word
    -DMB_Trailer2 before DMB_Trailer1
    Note:  DMB_Last_Word == DMB_Trailer2
    bit38: DDU Timeout Error (bad event, RESET req'd)
    *data from a fiber input either never started or never finished
    *an unknowable amount of data has been irrevocably lost
    bit37: DDU detected TMB or ALCT CRC Error (bad event, RESET?)
    *CRC check failed on 1 or more TMB/ALCT; possible one-time bit error
    *if error continues for consecutive events then RESET req'd
    bit36: DDU Multiple Transmit Errors (bad event, RESET req'd)
    *one bit-vote failure (or Rx Error) has occured on multiple occassions for the same input fiber channel
    bit35: DDU FIFO Full Error (bad event, RESET req'd)
    *an unknowable amount of data has been irrevocably lost
    bit34: DDU detected Fiber Error (hardware configuration change, RESET req'd)
    *change of connection status on 1 or more DDU fiber inputs; a hardware problem probably exists
    bit33: DDU detected DMB L1A Match Error (bad event, RESET?)
    *the DDU L1A event number match failed for 1 or more CSC boards; possible one-time bit error
    *if error continues for consecutive events then RESET req'd
    bit32: DDU detected CFEB CRC Error (bad event, RESET?)
    *CRC check failed for ADC data on 1 or more CFEBs; possible one-time bit error
    *if error continues for consecutive events then RESET req'd
    Note:  error bits requiring RESET persist until the RESET occurs.
    bit[31] = 0, hard-coded in DDU firmware
    bit[30:16] = 15-bits indicating which DMBs have data for this event, where 1 DMB == 1 CSC; one bit allocated per DDU fiber input
    bit[15:4] = HEX 000, hard-coded in DDU firmware
    bit[3:0] = number of DMBs which have data for this event


    Trail-2

    bit[63:0] = a 64-bit constant, hard-coded in DDU firmware:     0x8000/FFFF/8000/8000
    *this is a "DDU unique word" near the end of the event packet


    Trail-1

    decoded the same way as Header3 above


    Trail (S-Link64 DAQ Control word)

    bit[63:60] = 0xA       //hard-coded in DDU firmware by CMS directive
    bit[59:56] = 4-bits unspecified       //CMS directive, no details yet
    bit[55:32] = 24-bit DDU Word Count for the Event       //number of 64-bit words sent from DDU
    bit[31:16] = 16-bit Event CRC
    bit[15:8] = 8-bit Event Status       //from DDU; lower 4 bits by CMS directive, higher 4 bits are not assigned so I used them...
    *currently equivalent to Trail-1 bits [47:40]
    bit[7:4] = 4-bit DDU TTS Status       //CMS directive, current TTS status as reported to FMM
    bit[3:0] = 4-bits reserved       //CMS directive, no details yet
    *special "S-Link64 Protocol" status bits



    Link to Production DDU (small red DDU) Output Data Format.

    Link to old DDU Prototype Output Data Format.


    Revision Summary: