Guide to Data Acquisition at ATLAS
A Guide to Using the MSU/NSCL Data Acquisition System at ATLAS.
Here is a brief list of what's involved. More details will appear as we learn them. The goal is not to reproduce the comprehensive manual here, but to get the user started and provide information specific to the installation as it is configured at ATLAS.
There are several independent processes involved in acquiring, monitoring,
and recording on tape. The Front End is a VME based multiprocessor system. It replaces the event handler (and more) of the old system. There is a master which packs events into buffers and sends them out on the ethernet and a slave which talks to CAMAC and reads the data. The slave programming is done in C which makes it a much more versatile and powerful system than the familiar event handler language. To simplify matters even more many macros have already been written to simplify most of the CAMAC functions typically used.
The Back End is just a DEC ALPHA running VMS (oh well). It receives data buffers over the ethernet from the front end. A program called ROUTER distributes these buffers to CONSUMERS such as DAPHNE, SCALER, TAPE, and anything you may write on your own to monitor/analyze the data on-line. Currently, the standard histogramming task is DAPHNE. However, this is its sole task. It no longer handles writing data to tape, a distinction that is important to remember. If DAPHNE hangs up or crashes, your data is most likely still being put safely on tape by the MSU TAPE process.
- Wheel the front end rack (contains VME crate, terminal server) next to your CAMAC crates.
- Plug the Branch Crate controllers into the CAMAC crate(s) with the branch terminator plugged in next to the last controller to be daisy-chained. The order does not matter. Connect the branch driver in the VME crate to one of the CAMAC crate controllers using a branch cable (those big, unwieldly things). The pins are very fragile so be careful. If you have only one crate, plug the branch terminator into the second socket on the crate controller. If you have a second crate, connect the two CAMAC crate controllers with a second branch cable. Terminate the second crate controller with the branch terminator.
- Dial in the appropriate numbers for the crates. The branch driver in the VME crate should show 0 on its front panel. This is the branch number. The CAMAC crate controllers should each have a unique and appropriate number chosen using their front panel thumbwheel. The MSU system assumes you start at 1 and go up from there without skipping numbers (i.e. crate # 1,2,3,...).
- Connect the front end to the ethernet. If you are in the data room each data station has 2 ethernet taps with strange rectangular sockets that require special cables. Plug the terminal server into one socket and the VME ethernet interface into another socket; each gets its own cable. If you are in a target area with standard BNC connections, the two ethernet connections have to be daisychained by putting a BNC Tee on the input to each device and running RG58 from the panel-mounted spigot to the VME ethernet interface and then to the terminal server. If you are at the end of the ethernet branch put a 50 Ohm terminator on the last Tee in the chain. If you are not at the end, then continue the daisychain back to the second panel-mounted spigot. Keep in mind that when unplugging an active ethernet connection, you have ~15 sec to plug it back in before bad things happen.
- Turn on the terminal server. It will boot itself. You can plug terminals into its unused spigots to help you debug your electronics up close and personal. You can also use ports to talk to CAMAC crates via serial crate controllers.
- Turn on the VME crate.
- Make sure that nobody has unplugged any of the 4 terminal server cables form the fronts of the VME microprocessors.
- Make sure the ethernet transceivers (little grey boxes) are locked in place. The one on the front of the VME crate has a tendency to get loose.
- Set up the triggering logic. Refer to the section on electronics requirements for details.
- Log on to ANPH15 or GUPPY in the data room using the correct account.
- Open a DECTERM window by selecting from the session manager applications menu. You will control both the front end and the ROUTER-related processes from this single window. You can use 2 separate windows, one for the front end, the other for ROUTER, DAPHNE, TAPE, SCALER. But that is more confusing.
- SET DEF to the correct subdirectory.
- ATLASSETUP in this new window dedicated to data acquisition.
- Choose front end (ATLAS,PII,APEX). This tells it which VME setup to connect to on the network.
- Begin terminal sessions. This connects you to the VME microprocessor boards' terminal ports used to communicate with them.
- Select slave program. This is the C program you wrote and compiled with RCC. Enter without extension.
- Load front end processors. If ROUTER is running on the ALPHA and connected to the front end you want to load, you must kill it first since it will prevent the VME processors from halting properly before loading the master and slave programs. This is unlikely if you are starting from scratch. A cold start of the front end is needed only at power up as it says. You should get 2 'pROBE>' prompts before going on. If not, then go halt the processors manually by flicking the 'HALT' switches. Watch to see that the halt lights actually indicate the processors have halted. Then make sure the switches are back in the middle position. Hit a CR once you have ensured the processors have halted. Now a lot of loading happens. You are asked to hit a CR once in a while. Don't worry if it looks like nothing is happening. It is slow. You are not done until you get the SETUP menu back. If nothing happens for a long time, it is possible the processors did not actually halt.
- If you have a file called userfeinit.com it is executed at this time. It is useful for setting such things as scaler parameters, connection status, tape status, userint values, etc.
- Exit to DCL.
- @filename.SETUP to load DAPHNE. Do all the usual DAPHNE things for sorting data off-line. This is not a DAPHNE tutorial so consult the standard sources of DAPHNE information. A few items make this application different. The userfunction is linked using @DAPEXE:NEWMSUUSERFUNC.LNK usrfuncname. Most importantly, DAPHNE will not be writing to tape. It is used only to monitor the data.
- Start a graphic display. Consult the DAPHNE section to choose from the available options.
- ATLASSETUP.
- Determine scaler display option. It is either ANPH15 or GUPPY. In addition, the terminal server port used by the scaler terminal should be written on a label.
- choose start ROUTER/NOTIFIER option. You will get a message about forming a link to the front end.
- If you have a file called userstartup.com, it will be executed at this time. It is a good way to automatically make the connection to DAPHNE and scalers.
- If you answer 'y' to scaler display, enter .SCA name w/o extension. But
first make sure you do not have a 'Local' prompt on the scaler display
terminal. If you do, then type LO or CTRL-Z to log out of the terminal server on the scaler display keyboard.
- After you see the scaler display terminal respond, type D on scaler terminal, answer the question, hit [CR].
- Hit [CR] in the workstation data acq window if it is waiting around.
- Exit to DCL
- TAPESTART
- first tape drive select (i.e. mka500:)
- 1-drive program
- ignore messages issued by the system here
- Controlling the run
- SRUN x (automatically incremented, but forgetten in crash)
- TITLE "put anything you want here"
- TAPE ON (you choose on or off)
- BEGIN (begins data collection)
- ACQ
- beginning:
- TITLE "something informative" if it needs changing
- TAPE ON (or OFF) if it needs changing
- TAPESTART if new tape and not already done, select drive if none is listed, choose 1-drive program
- BEGIN
- there is some delay before it really starts and you get the prompt back. A good way to tell is to see when the scalers reset and begin counting again.
- ACQ if you stopped it
- ending:
- END and write down # buffers written to tape
- to change tapes:
- STOP TAPE (DCL command to kill process)
- DISM mkaxxx:
- label and insert a new tape. It does not need to be initialized.
- any DAPHNE stuff is optional: STP, ZAP, WR/1D, INZ, etc.
- monitoring:
- keep an eye on the scaler display. Is it counting and making sense?
- look at any appropriate DAPHNE spectra, zeroing some peroidically to make sure the detectors are still working.
- SHOW SYS in any window to make sure no process has been suspended.
- keep an eye on the LCD tape counter. Don't get too close to the end.
- CNT in DAPHNE will tell you how much of the data is actually being sorted on-line.
If the data rate is too high, the front end has a tendency to crash unexpectedly. To make matters worse, it may do so quietly. If there is no system dump in the data acq window, then the ways to tell are by noticing that the scalers have stopped increasing (you have to be watching the 'totals' of course), or your spectra have no new counts. Since the system seems capable of crashing often (usually a result of user errors in the triggering electronics or slave program), it seemed only natural to have a separate section on what to do after. Some things will still be loaded and/or running. Some will not. Some will have to be killed even if they are running.
Sometimes only DAPHNE crashes and takes the entire window down with it. This has the unpleasant side-effect of killing the ROUTER, tape, and scaler processes. This will stop your data collection since there is no process writing to tape. If DAPHNE hangs up, but the window remains alive, then data is probably still being collected. The trick is to figure out how to unhang DAPHNE without disturbing data collection.
Below are lists of what you should do if the front end crashes or if just the window gets blown away without a trace (or has to be killed voluntarily).
- Workstation Crash:
- Reboot if it has not already done so by itself. There is a little button on the back of the workstation. Push it and then type B at the prompt.
- Log on to the correct account. Hopefully you know the username and password.
- Read the Step-By-Step Computer Setup Procedure to see what to do next. You should probably even reload the front end because the ethernet interface tends to get hung up if things crash while running.
- Front End Crash:
- type GO at the pROBE> prompt to see if that restarts the slave. If not, you have to reload the processors.
- You might as well do an STP in DAPHNE.
- ATLASSETUP
- Abort ROUTER (TAPE and SCALER then die too)
- ATLASSETUP
- Load Front End Processors. Make sure they halt! (see 2 'pROBE>' prompts)
- Exit to DCL
- ATLASSETUP
- Start ROUTER
- exit to DCL
- Dismount old tape
- Label and load a new tape
- tapestart to start TAPE
- SRUN x
- TITLE "whatever"
- TAPE ON
- SHALL to see what state the front end is in and if anything is set wrong
- BEGIN
- ACQ
- Window Disappears:
- select DECterm from application menu in session manager.
- ATLASSETUP
- Choose scaler display option
- Select front end
- Begin terminal sessions
- Select slave program
- You should not have to reload the front end.
- check the front end status (SHALL) window to see if tape is enabled, if run # is correct, if run is halted.
- start ROUTER, answer scaler startup questions (y to scaler startup and enter .sca name)
- Exit to DCL
- TAPESTART, choose drive, 1-tape program, exit
- @filename.SETUP to start DAPHNE
- restart any extra subprocesses like the display, postscript plotting, automatic monitoring, etc. Consult the DAPHNE section for details
- BEGIN run
- ACQ to start on-line sorting again.
All the usual DAPHNE structures, commands, etc are unchanged for the most part. Consult the usual sources of help if necessary. Only those commands which are modified or might cause confusion are mentioned.
- make daphne (create stream of data called daphne)
- send 1/sample daphne (sends data buffers to daphne stream in sampling mode)
- ACQ (begin getting data buffers from ROUTER)
- stp (stop sorting into DAPHNE spectra, data still collected to tape)
- cnt (the usual stuff plus % of TAPE data sorted by DAPHNE)
- ddv x (for crude X windows display on workstation)
- ddv/xnew (for new and improved X windows display on workstation).
- Users can now over-ride the default color tables in the file
dapexe:xdsp_colors.colors by redefining the logical name "XDSP_COLORS" in the
job logical name table.
- The do1h option now works properly.
- It appears to cause the AXP/VMS 6.1 windows manager to fail the
second time it is run. (The first time it is run on AXP/VMS 6.1 it works
ok). This problem does not appear in AXP/VMS 6.2 and the cause is unknown. The
window manager can be restarted by users (even without privilegeds) by using
the command: @sys$manager:decw$mwm.
- @dapexe:tgraf (for tektronix-like display on workstation screen)
- ddv/type=4115 17bx (for tek terminal using GRAPH at PHYLIS prompt)
- ddv/type=4115 LTA500x (for tek terminal connected directly to terminal server). Make sure you are logged out of the server, i.e. you do not have a 'Local' prompt on the tek terminal. If you do, type LO or CTRL-Z.
- PDV ATLAS_POST (usual plotting subprocess)
- PDV ATLAS_HP1600C (color printer)
- KLP to kill plotting subprocess before starting a new one
- INZ information is not written to tape so don't worry about keeping it up-to-date unless you want it correct on any plots you make on-line.
- If you use fixed-length events in DAPHNE and have included an end-of-event marker in your slave readout program, remember to add 1 to whatever you think your fixed-length is.
- If DAPHNE hangs up or crashes, do not worry. Just kill all its processes and restart it...assuming the window is still there. It is wise to keep and up-to-date .setup file around in case the inevitable happens. Restarting will not disturb data collection unless the relevant tape processes or the entire workstation crash. If the window dies, refer to the section on restarting after a crash.
- Remember that any Event processor routines you used on the VAX with the multibus must be turned into standard fortran.
- The Debugger! This will greatly reduce the time it takes to find mistakes in userfunctions. Compile using FORT/DEBUG/NOOPT filename, link using @NEWMSUUSERFUNC.LNK filename /DEBUG, and begin sorting with ACQ/DEBUG. The menu driven symbolic debugger pops up. Just click on GO and wait for the error. If you are scared of the debugger, at least use some write statements in places to help locate errors.
- If you get arithmetic traps in your userfunction, the debugger won't easily help you find them since daphne takes care of them. It aborts the userfunction at the point where the trap occurs and does not histogram the event. If you put write statements after key points in your userfunction, you can see how far it gets before the error. This will help you zoom in on the correct line.
- For those of you who miss the old EPFIFO program, but do not like the front end's primitive buffer dumping capabilities there is a program to dump single events as they are received by ROUTER. To use it: RUN DAPDIR:CDDUMP or there is probably a symbol called EPFIFO for old time's sake.
- You must include scaler reading and clearing in your slave program. They are cleared in usrbegin and usrrdscl. They are read in usrrdscl. Don't worry that they are cleared after each read. The display program keeps track of running totals among other things. You can read all channels of a scaler simply by using the
READALLxxxx
macro (xxxx is the scaler model #: 2551 or 4434). Consult a sample slave program to see where the lines go in the code.
- Set some reasonable values for the front end to read the scalers (FREQ) and to write them to tape (FRACTION). Don't worry about the scalers being zeroed after each read. If you read scalers every 10 seconds, but write them to tape only every minute, the front end keeps track of the total for the time interval between tape writes.
- Write a .SCA file to tell the scaler display program what signals are plugged into which channels and what you would like displayed on each line. Print out the example or at least look at it on-line because it has many comments in it which are not repeated here.
- If you do a READALL in 16 channel scalers, you must treat the last 4 channels as the first 4 of a second scaler in your .SCA file since the display program knows only about the LC2551 12-channel scaler. See the sample .SCA file to see what to do. In the .SCA file, the slot # and channel # do not refer to CAMAC slots and addresses. The scaler display program deals with buffers of numbers. It does not know what modules they are coming from. It assumes the first 12 numbers are from scaler #1, the next 12 numbers are from scaler #2, etc. The first # after the word
INPUT
is this pseudo module #. The next # on the line is the pseudo address in that pseudo module. They are numbered from 1 to 12, not 0 to 11. All this numbering would make perfect sense and cause no confusion if we used only 12 channel scalers such as the LeCroy 2551.
- SSCAL and answer questions, or do each command by yourself if you remember the relevant ones.
- tot # of scaler channels read out by slave program (Important)
- frequency to read scaler (# seconds between reads)
- tape interval (freq*interval = frequency of writing to tape)
- ATLASSETUP and choose scaler display option. Select ANPH15 or GUPPY. This method replaces the original way which used GRAPH lines on PHYLIS. It has the advantage of being dedicated and not requiring redefinition if SCALER crashes during a given login session.
- You must have a .SCA file which contains scaler information. It tells the
display program what is in the scaler buffer and what to display on the screen.
- You must make sure you do not have a 'Local' prompt on the scaler display
terminal. If you do, type lo or CTRL-Z on that terminal keyboard.
- SCALERSTART. This runs a .COM which does the following:
- @daq_scaler:sclstrt filename (whatever .sca file you want to use)
- don't worry about messages regarding problems connecting.
- answer questions on scaler display terminal. Type D to select a device to which you want scaler data written at the end of each run. Enter a filename or NL: to turn off end-of-run summaries. Hit a [CR] when done.
- a page of defined scalers appears, with 0 counts of course. If no names, or wrong names appear, there is a mistake in the .SCA file.
- hitting [spacebar] on the scaler terminal toggles between rates and total counts.
- the rate is in counts/sec. The program takes into account the frequency of readout.
- hitting [1], [2], etc. switches to that page if you have multiple pages.
- STOP SCALER kills scaler process. To restart, you simply type SCALERSTART and go through the normal routine. The scaler display option should still be defined unless you had to create a new window. The old way of selecting a GRAPH line on PHYLIS is no longer necessary.
- There are 2 types of files written at the end of each run: .SCL and .TXT. The first is just the raw numbers at the end of the run. The latter is a snapshot of the scaler display page showing totals at the end of the run.
- Remember to run this after ROUTER is started.
- TAPESTART (.com file to start tape process on alpha)
- choose drive (MKA500: for 4.7Gb, MKA501: for 2.4Gb). These are both names for the AVIV drive with the nice LCD display. It is best to use this drive if you want to know where your tape is at all times.
- Choose single or double density 8mm. This affects the little on-screen counter and when TAPE thinks it is at the end of the tape. The actual drive density is determined by the name. However, the workstation may crash if you select single-density 8mm, mount a double-density tape drive, and try to keep writing past 2.2Gb (normal end of single-density 8mm).
- start 1 drive program.
- ignore message about illegal request.
- When you BEGIN a run, you may get many messages from ROUTER about running out of input buffers if the data rate is very high. This is because the tape takes time to get moving and the 'fe' is sending buffers to ROUTER during this time. ROUTER has only a set number of buffers to accumulate data in. If TAPE is not yet emptying them, then they fill up since TAPE is not run in sample mode. It takes all the data.
- Don't forget to enable writing to tape (TAPE ON).
Many front end commands can be issued both from the standard DCL prompt ($) as well as from the front end's own prompt (RcTL>). The commonly used commands like BEGIN, END, etc. have DCL symbols which will execute them without explicitly connecting to the front end first. Ignore this comment and some details below about 'connecting to the front end' if it confuses matters. Just stick to the DCL commands.
- Write a slave program (sample1, sample2) to read out your CAMAC modules. Check out the section on Slave Programming for some insight and suggestions based on many lessons learned the hard way. RCC filename compiles and links it on anph14 where the compiler resides. It spews out a long list of messages which are irrelevant as long as you get messages at the end saying "0 errors." Beware that the compilation may be slowed down by heavy usage on anph14.
- TAPE ON/OFF (enable/disable writing to tape. DON'T FORGET!!)
- SRUN run# (sets run # in 'fe'. incremented automatically!!)
- TITLE "put it in quotes like this" (set run title put on tape)
- BEGIN (activate the front end to send data to the ALPHA)
- END (halt the front end, no more data sent out, EOF is put on tape)
- Be careful you don't get confused and think begins a run because it just starts DAPHNE sorting. No events will be coming in or going to tape.
- STATUS (displays things like whether run is halted or running, whether tape is enabled or disabled, whether 'fe' is connected to the outside world or disconnected, current run #, title, and more)
- SHALL (a complete list of all relevant 'fe' parameters)
To gain access to the front end, type fe at the DCL prompt. You then have an 'RcTL>' prompt. A complete list of commands and the general syntax is in the front end section of the MSU manual. Below are some commands which must be executed from this prompt.
- BCNAF branch crate address function data (allows individual cnaf commands to be sent to CAMAC modules which are under the control of the 'fe'. The branch is usually 0 since we have only one. The data is optional and needed only for writes. By the way, don't use commas.)
- SET DUMP ENABLE (starts event buffer display)
- SET DUMP DISBALE (stops it, can type while data is streaming by you)
- SET DUMP WORDS 128 (determines the # of words to be dumped at once)
- It is wise to put some easily visible marker at the end of your event to make the dump clear. 0xDEAD seems to be popular. Just put it at the end of your slave program's readout section. Every once in a while you may see weird stuff in the dump. It is probably a scaler buffer.
- The first word is the number of words in the event (in HEX of course) including itself and any end-of-event-flag you may have included.
- SET INTEGER array_index value ('fe' has downloadable parameters like FIX/FLT in DAPHNE. Remember that array indeces begin with 0 in C.)
How to reload the front-end with minimal disruption
- END the run
- STP daphne sorting
- STOP TAPE (Unfortunately you will need to start a new tape)
- Abort ROUTER in ATLASSETUP. Otherwise front end won't load properly.
- Make any changes you want to your slave program and save it.
- RCC slavename and make sure there are no errors
- If the program name has not changed, choose the load option in ATLASSETUP
- If the name has changed, select slave program in ATLASSETUP before loading
- If the VME crate has not been turned off at all, say no to a coldstart
- Hit a CR if you get the 2 'pROBE>' prompts and wait for the front-end to load
- You might want to watch all the messages to keep an eye out for errors or irregularities (assuming you would recognize them)
- SRUN to reset the run # to what it was before
- TITLE to reset the title to what it was before or make up a new one
- TAPE ON if you want to take data to tape from the start. Otherwise wait.
- Start ROUTER and scaler display in ATLASSETUP
- TAPESTART
- BEGIN to begin acquisition
- ACQ to start daphne sorting
- When (not if) the front end crashes, there will be a stack dump in the data acq window followed by a 'pROBE>' prompt. This does not necessarily mean the front end has to be reloaded. The run can usually be resumed simply by typing GO at the 'pROBE>' prompt. You should either be returned to DCL or be able to type ^Z to get there. If you did something in the data acq window after the dump and no longer have the 'pROBE>' prompt, then type CONNECT/TERM/CHAR 68K_SLAVE to get it. Now type GO followed by a ^Z to get out. Do a SHALL to make sure things are OK. If there is no response, the front end is truly dead. Consult the section on starting up after a crash.
- If a process seems hung or has problems, do a SHOW SYS to see if it is in the MUTEX state. If so, it is swapped out and may have trouble getting back into memory. Try killing some non-essential proceses. Otherwise you have to bite the bullet, kill the process, and restart it. Too many sub-processes in a single workstation window may cause this to happen. A good place to start stopping processes is:
- second DAPHNE display (KLG/SEC).
- plotting process (KLP).
- scaler display (STOP SCALER)
- main DAPHNE display (KLG).
- any monitoring subprocess you spawned to periodically show spectra
After the important processes return to normal, you can try to restart those which you killed. The system usually accomodates them.
- The run is active, everything seems like data should be coming in, but the system is inhibited. This can happen if you unplug the event signal or some part of the busy possibly. Since the busy gets cleared only at the end of an event, the program is likely to miss clearing the BUSY latch if you pull the INT2 signal. Just push the STOP button on the BUSY latch to get things going.
- You have an ADC which seems to forget its control register (or whatever it may be called in your module) values. The routine usrbegin calls usrini first thing at the beginning of every run. usrini does a crate Z which causes many modules to forget things. Just stick some camwrite16's at the end of usrbegin (after usrini is called) to set the control registers. You can even make the values adjustable by using the front end's SET INTEGER feature.
- In a dump following a 'fe' crash, addresses beginning like FA8... indicate some sort of CAMAC error.
Bus/Address Error. Address=FA8111AE Running: 'EVTS' -#002823D0
--------------------RunCtl> -------------------------------------------------
SR=0004-tfsm.000...xnZvc USP=001CF670 MSP=002FDFF0 ISP=00281074
VBR=00000000 SFC=0 DFC=0 CACR=11 CAAR=00000000
DR=002000E2 00000014 00000006 002000D8 00000402 00000000 001CF6CC 002000D8
AR=002000E2 00009DC6 002000C8 002000E4 00000014 FA823812 001CF694
PC=00004244-00004244 4A79FA8111AE TST.W ($FA8111AE).L
This example has an address of FA8111AE which can be decoded with a simple program Ken Teh wrote. Just do RUN U19:[TEH.DAQ]ADDR and enter the address. It will return the B,C,N,A,F which was being executed and most likely was the cause of the crash. This one happened to be from a command in the slave program which tried to talk to an empty slot in a CAMAC crate. Another example is trying to talk to the 12th channel of an 8-channel device. Its effect is similar to trying to read/write an array out of bounds. Check your slave program for typos. If the decoded CAMAC address and function seem to be valid, try putting a donithing(); line in the slave program just above the line which causes the error. There may be some subtle timing problem.
- Another type of front end crash dump might look like:
FE-F-PANIC FE-F-BUG Unrecognized front end event type status = 0x0
Temporary Breakpoint. Running: 'BFMT' -#00288C10
---------------------------------------------------------------------------
SR=0004-tfsm.000...xnZvc USP=001C846A MSP=002FBFF2 ISP=002820C8
VBR=00000000 SFC=0 DFC=0 CACR=11 CAAR=00000000
DR=00000000 0002999A 001C8496 00000FF0 00002000 0028223E 001C8572 00000000
AR=0002999B FC000804 002763EC 001C647A 001C647A 0000A0FA 001C847A
PC=0000236C-0000236C 4E4F TRAP #15
This one might indicate a bad CAMAC branch driver (the VME module). The FE-F-PANIC FE-F-BUG and TRAP #15 parts of the message are the things to look for. If there is a hardware problem, it seems to become most apparent at high rates. If possible, try lower rates or maybe borrow a different branch driver.
- Messages about "redundant buffer pkt ignored" scrolling across data acq window. These are the result of ethernet collisions and can be ignored if they are few and far between. If they keep happening, you have a problem. One cause is the MUTEX problem referred to above. ROUTER has probably gone into this state. The solution is the same as above.
- When starting TAPE, you get a message about an illegal request being ignored. You did not have the privilege to for TAPE to increase its priority. Ignore this message.
- You get a message about losing the link to the front end. The link usually reforms all by itself. Wait a few minutes. If it does not reform, you will have to abort ROUTER and then take care of all the things associated with restarting ROUTER. This includes starting a new tape so be patient before taking drastic measures. At low data rates, data will not be lost because the 'fe' can buffer it to some extent. At high rates, the 'fe' will stop taking data if it cannot send it or buffer it.
- BCNAF complains about insufficient parameters when you try to send a command. It probably needs data. Higher numbered function codes are supposed to be writes in the CAMAC standard. If you have a module which is using it otherwise, you must still supply data (just a 0).
- Of 16 channels in a module, only 8 seem to be there. Or in a dump, channel 1-8 repeat over for 9-16. The 16th bit is missing. Before you consider blaming a flaky crate controller, see if you merely failed to tighten the branch cable connectors properly. You might get lucky. If not, swap controllers. The next step is the crate, assuming the modules are not at fault.
- Your userfunction crashes with some unhandled exception. Do you have fixed-length events? If you have included an end-of-event marker in your readout, have you forgotten to add 1 to your fixed-length event size in DAPHNE?
- When you do a SET DUMP ENABLE to view your data stream directly in the FE terminal window, you don't see end-of-event markers when you think you should. You may have a typo in your slave program which causes many more words/event to be written out. The end-of-event marker then requires a larger SET DUMP WORDS.
- Your front end seems to be collecting data (lights flashing, etc.), but nothing appears on the scaler display or maybe nothing gets histogrammed by DAPHNE. Take a look at the CAMAC crate controller(s) and see if the Inhibit light (I) is on. If so, then modules will not be read properly. The inhibit should be cleared whenever you begin a new run. If you are running and a crate is still inhibited, then try to clear the crate manually by sending the proper BCNAF command. Consult the section on CAMAC Hardware Setup/Communication (CNAF's).
- Your histogrammed data and scaler display make no sense. The scalers continue to count even after you END your run. This is a really confusing error which can now occur with the presence and possibly simultaneous use of both front ends. If you connect your ROUTER to the wrong front end, you just might get some data and scalers counting if another group happens to be running. That is why doing things to your front end has no effect on your back end operation.
- DAPHNE gives a message referring to global sections. It has most likely used them up and the only way to remedy this is to kill the window and start over. This has the unfortunate side-effect that you must interrupt data collection because killing the window kills the ROUTER. To save time you can open a new window, get DAPHNE loaded there, then kill the original window. Remember that DAPHNE won't let you use the same process name as the other window so type SET PROC/NAME=JUNK in the original window before trying to start DAPHNE in a new window. Refer to the section on how to restart after a crash for the steps to restart everything in the data acq window. Delete all the old .sec files when you are done.
- You have trouble loading the front end. It gets part of the way through the process, gives some error message, and seems to continue. Don't be fooled. If you have one of the BiRa CAMAC crate controllers, see if the front-panel ACL light is on. If not, either change the jumper on the side of the module or connect the REQ to the GRANT IN on the front of the crate controller.
Two weaknesses of the interrupt-driven MSU system require some hardware modifications to any standard setup you may already have: 1) It cannot generate a busy signal until ~25 usec after the event signal (INT2) has been received. 2) It cannot wait a calibrated time after being triggered for things such as ADC conversions. We have drawn a schematic of how to implement this trigger logic.
The optional delay for the event signal replaces the software delay we had in old event handler codes. It's value should be chosen such that there is enough total delay for adc conversion before the program begins to read out. If you need 80 usec for an AD811, for example, then add ~60 usec delay to the event signal. If none of the electronics needs more than 25 usec conversion time, then there is no need for any delay.
The width of the event signal plugged into INT2 should be at least 100 nsec. There is no reason to skimp, and many reasons not to. Double-firing of the event signal may send a second interrupt to the slave processor before it has had a chance to process the first one and block out further ones. This may cause the slave to crash.
The 222 gate generator used in latch mode performs the function of setting a prompt busy. Simply start it with the UNDELAYED event signal. The MSU system will generate an end of event signal which is available via a BiRa output register (chan # 1) in the camac crate. We use this signal to end the busy signal. It can be used for other things you normally like to do at the end of an event. In the BGO setup we also use this end of event signal to clear our MLU camac module.
A consequence of using the latching gate generator is that unplugging the event signal and plugging it back in may leave your electronics inhibited since you will have started the BUSY latch, but not necessarily stopped it. Just push the stop button on the LeCroy 222 to get acquisition going again.
The MSU system also provides a "STOPPED RUN" signal. It sets a bit in the BiRa output register (chan # 2) at the end of a run and clears it at the beginning. Just fan this in with the normal BUSY to inhibit the electronics at the end of a run.
- Check that the correct number of CAMAC crates is specified. Near the top of the slave program
syscrates{NUMBR] = {2};
is declared. If you have only one crate, change the 2 to a 1.
- Use #define at the very top of the slave program to define module slot and channel assignments to avoid having to change numbers in multiple parts of the program.
- You can even use arrays if things are in oddly spaced slots.
- Of course, you have the full power of C at your disposal, but certain structures are slower than others. If you are concerned about having the fastest event readout possible, then keep this in mind. For example: 'for' loops are much slower than just doing something repeatedly. I had a 'for' loop to read 4 adc's. Replacing it with 4 reads saved 20 usec.
- As a diagnostic tool to see where your program spends most of its time, use the output register to create pulses at various times during the readout cycle. Consult the slave program to see how this is done. There are already some pulses generated such as the one to clear the busy latch.
- To help debug your slave program you can insert
fprintf
statements anywhere you like. For example: fprintf(stderr,"bufpt set to %i\n",(int)bufpt);
would print the value of bufpt for each event. You have to connect to the front end using the fe command, BEGIN the run, and watch the numbers scroll by. It is a useful tool if you are not sure that certain modules are actually being read or cleared properly and a dump is too confusing.
- You can program the output register to set bits or create pulses for all sorts of useful jobs like clearing many adc's at the end of an event with a single command instead of several.
CNAF commands can be issued in the MSU system using the
BCNAF command from the front end (FE) or the 'RCTL BCNAF'
command from the regular command line. However, there is
currently no software ready to do these tasks from inside
programs. Fortunately, the branch (parallel) crate
controllers, BiRa 1302LM A-2, used in the MSU data
acquisition system allow us to use our old Kinetics system
controllers , KS3989 RS232, as 'auxiliary' crate controllers
in the CAMAC crates. This allows us to issue cnaf commands running
the same programs which we used in the old DAPHNE system
- via these auxiliary crate controllers.
To use the KS3989 controllers as auxiliary crate controllers
16 resistor packs must be removed from them. Six of them are
easy to get access to; but 10 of them are only accessible by
removing one of the PC boards of the module (see the manual for
instructions on how to do this). Some of the resistor packs
are removed so that the auxiliary crate controller does not
terminate the CAMAC bus - that should only be done by the
master crate controller in slots 24/25. Thus, only modified
KS3989 controllers can be used as auxiliary crate
controllers. Likewise, modules that have been changed cannot
be used as master crate controllers anymore. The modules
that have been modified are marked as such with yellow tape
on the side of the boxes. Please clearly mark any that you
may change, and do NOT lose the resistor packs!
The main crate controller (the BiRa) in slots 24/25 must
have a cable from its 'REQUEST' to 'GRANT IN' LEMO
connectors. In addition, a cable must connect the BiRa crate
controller's 'GRANT OUT' (in slots 24/25) to the KS3989
controller's 'GRANT IN'. The auxiliary crate controller
(KS3989) can be placed in any slot of the CAMAC crate. Since
some of the CAMAC control lines are only available in slots
24/25, a ribbon bus cable (called "ACB" or Auxiliary Crate
Bus) connects the master crate controller to the auxiliary
crate controller. The 40 pin bus connectors are on the back
of the crate controllers. When the auxiliary crate
controller receives a CNAF command over it's front-end RS232
serial line, the auxiliary crate controller will issue the
command from the main crate controller via the ACB bus. It
doesn't actually do it itself. The request/grant lines are
only there to establish the priorities of the crate
controllers, all communication is done over the backside ACB
bus.
figure: Auxiliary crate controller connections.
As opposed to the DAPHNE system, the RS232 lines used to
issue the CNAF commands are not direct lines from the DAQ
computer to the CAMAC crate serial controllers. Rather,
ports 2,3 and 4 of the terminal servers, which are part of
the MSU rover DAQ systems, are used for this purpose. You
can access those lines from the two ALPHA DAQ machines as:
- lta8302: port 2 of the ATLAS DAQ rover station
- lta8303: port 3 .....
- lta8304: port 4 .....
- lta8102: port 2 of the PII DAQ rover station
- lta8103: port 3......
- lta8104: port 4 .....
- lta8332: port 2 of the NADA DAQ rover station
- lta8333: port 3......
- lta8334: port 4 .....
All the lines are accessible from either of the two ALPHA
computers.Please note: These lines used to be called
lta8412,....lta8514. They were changed in August 1996
Ports 5-8 of the terminal servers are used for the MSU
system processors in the VME crate. Port 1 can only be used
as a 'local terminal'. However, ports 2-4 may be used either
as 'local ports' (i.e., hooked up to dumb terminals) OR be used as
RS232 lines for the DAPHNE cnaf commands.
WARNING:
- Some CNAF commands do not execute when the data acquisition
system is running. It's best to be stopped (issue an "end" in
the FE window).
- The data acquisition system may leave the CAMAC crates
in a mode where the CNAFS cannot be issued. It may be necessay
to reset the CAMAC crate as:
cnaf lta8512:
1,30,0,17
1216
1,30,1,16
1,30,13,17
This example is using the second port of the PII system and
is resetting crate #1
Some notes:
- It may be necesary to issue a CVD command from the daphne
system after the terminal server has been re-powed:
dap xxx
cdv lta####
This will set some characteristics of the terminal server port.
- occationally the terminal servers are changed - in which
case the lta numbers change too. To see what lta lines are set up
for what terminal server, consult the LAT$SYSTARTUP.COM file
in the SYS$MANAGER directory. You can check what terminal server
you are dealing with from the console (port 1) by typing SHOW SERVER.
- Some of the most important characteristics of the terminal
server (to function) are
- port: remote configuration
- port: port access dynamic
- server: incoming logins: Telnet
(there may be more; but these are the ones I think are crucial).
-
To test the CAMAC talk lines on the MSU terminal servers one can do the
following test:
Plug the MSU terminal into port 2 (or whatever port you are using). Then
hit Ctrl Z a couple of times until the terminal server responds with a
"exiting the.....". If the port is not in that mode it will not receive
anything. Now go to one of the VME machines in the data room and issue
a CNAF command to the port. You should see some gibberish
on the screen, confirming that it is communicating. If you put the terminal
in the mode where it displays control characters rather than act on
them AND you drag Phil Wilt down in front of the terminal, you can even learn
what instructions behind the CNAF commands are.
A number of lat ports have been set up on the terminal server
in the data room for scaler and DAPHNE display terminals.
lta5001: scaler display on terminal next to anph15
lta5002: scaler display on terminal next to guppy
These connections are usually not touched and selected automatically
from the ATLASSETUP scaler display menu entry.
A number of lat ports are set up for DAPHNE display terminals. These
connections are somtimes changed - so check the connections if you cannot
make the displays work. The standard connections are:
lta5003: connected to the 4115 display @ anph15 (19200 baud)
lta5004: connected to the 4107 display @ guppy ( 9600 baud)
lta5005: connected to the 4107 display @ anph15 ( 9600 baud)
To make the DAPHNE display connection to e.g. the 4115 terminal:
hit ctrl-Z on the 4115 so that it no longer shows the "lat-3" prompt.
ddv/type=4115 lta5003:
DISCLAIMER
Originally written by Daniel Blumenthal,
now maintained
by [email protected] (Torben Lauritsen x4026)
$Id: index.html,v 1.2 2000/08/08 15:37:21 tl Exp tl $