How WebDAQ works

This chapter explains what the system does when you use it — where the data goes, what a run consists of, what is stored and what survives a restart. It is about behaviour, not code; for the code, see the Server Architecture Guide.

1. The data path

CAEN digitizers  ──►  CaenDAQ (in process)  ──►  data/run<N>/*.caendat
                             │
                             ├──►  live spectra and waveforms  ──►  browser
                             └──►  rates  ──►  Graphite  ──►  Grafana

Acquisition runs inside the DAQ server process. The boards are opened, read, decoded and written to disk by CaenDAQ, a C++ library bound into Python. There is no separate acquisition process, no container to start, and no network hop between the boards and the file.

This matters in practice:

Nothing to start or restart separately. Pressing Start is the whole operation.
Online monitoring costs nothing extra. The spectra you see are the ones CaenDAQ is already accumulating in C++ as it decodes; the server takes a snapshot on demand. No second copy of the data stream, and no monitoring process to fall behind.
If the web interface dies, the run does not. The frontend is a separate process that only talks to the server over HTTP. Closing the browser, reloading the page, or restarting the frontend has no effect on a run in progress.

Note

Earlier versions ran acquisition as XDAQ inside a Docker container, with a TCP “spy” socket for monitoring. That is gone. The file format is unchanged — .caendat files keep the exact XDAQ header layout on purpose, so runs from any version of WebDAQ convert with the same RUReader.

2. Starting from nothing

You do not create a database, a user, or a directory layout by hand.

Run LunaDAQ and open http://localhost:3000.
Click Start an Experiment and choose a working directory.
The DAQ server is launched in that directory and creates what it needs: conf/, calib/, data/, an app.db database and a default user.

The working directory is the experiment. Everything the server reads and writes is relative to it — board configurations, calibrations, run data, the histogram dashboard, the accumulated charge. Two working directories are two independent experiments that happen to share an installation. Starting the server from a different directory gives you a different setup, not a broken one.

This is worth knowing when something looks “missing”: a dashboard with no histograms, or a board list that is empty, usually means the server was started somewhere other than where you configured it.

3. Adding the boards

Boards are discovered, not typed in. Board page → Scan.

The scan walks the connection types the CAEN drivers support — USB, optical (CONET/A4818) and VME — and reports what answers. Add a board and WebDAQ reads its full register set from the hardware and stores it as one JSON file per board in conf/, named after the model and link (V1730_0.json).

From then on that file is the board’s configuration. It is written when you change a setting in the UI, read when the server starts, and copied into the run directory when a run begins, so every run carries the settings it was taken with.

Board settings — thresholds, trapezoid parameters, PSD gates, coincidence windows — are covered in CAEN digitizers.

4. What a run is

Pressing Start does this, in order:

Freezes the current board configurations and copies them into data/run<N>/.
Hands the open digitizers to CaenDAQ, which arms them and starts them together using the boards’ own synchronised start.
Opens data/run<N>/ and begins writing .caendat files.
Switches on everything that follows the run: charge integration, the rate collector that publishes to Graphite, the stats recorder, and any run-linked Grafana alerts.

Pressing Stop unwinds it, and additionally writes the things that describe the run: its accumulated charge, its ROI counts, its metadata row, and the logbook entry if you asked for one.

Run numbering increments automatically when data saving is on. You can set the number by hand from the status card — click it — which is what you want after a spoiled series.

Data saving off is a real mode, not a mistake: the run proceeds, spectra and rates update, charge integrates, and nothing is written to disk. Use it for tuning.

If a board fails mid-run

A board failure raises whatever alerts are configured for it (Settings → Notifications, Telegram and/or Zulip), the Board Health tab shows the board’s counters — blocks read, dropped blocks, read errors — for the run, and the Troubleshoot page keeps the event with the fixes offered next to it.

With auto-restart enabled, a board that stops responding causes the run to be stopped, flagged bad in the logbook with an explanatory note, and a new run started automatically. The new run continues the failed one: its beam-current file, charge and stats.csv are its own, and it carries over the target, the terminal and probe voltages and the run type. Run-linked Grafana alerts stay active. The failure is written to the Troubleshoot history whether or not a rule was configured to deliver it, and an alert goes to Telegram or Zulip if one was. The setting and its delay are kept in conf/settings.json, so they survive a server restart. The intent is that a night shift loses one run rather than the rest of the night.

5. Changing settings while running

Some settings can move during a run and some cannot, and WebDAQ enforces the difference rather than letting you find out the hard way.

Setting	During a run
Thresholds, trapezoid parameters, PSD gates	Yes — see online tuning
Rebinning, log scale, ROIs, dashboard layout	Yes — display only
Rate sampling cadence	Yes, applied from the next run
Beam current module and its settings	Yes
Adding or removing a board, sync mode	No — stop the run first
Run number, data saving	No

Online tuning is the useful one: open the tuner, change a threshold, and watch the trace and the spectrum react. Only the registers that are safe to write while the board is armed are sent; the rest are stored and applied at the next run. This turns filter tuning from a stop-change-start loop into something you do while watching the peak.

6. What is stored, and where

Everything is under the working directory.

Path	What	Survives restart
`conf/<board>.json`	One file per board: its full register configuration	Yes
`conf/settings.json`	Run number, data saving, size limits, the board list, auto-restart and its delay. Written atomically under a lock, and kept rather than replaced if it becomes unreadable	Yes
`conf/histograms.json`	The histogram dashboard: which spectra, their ROIs, zooms, layout	Yes
`conf/current.json`	Beam current module, its settings, lifetime accumulated charge	Yes
`conf/stats.json`	Graphite server, metric subtree, monitored metrics, sampling cadence	Yes
`conf/telegram_settings.json`, `conf/zulip_settings.json`, `conf/elog_settings.json`, `conf/grafana_settings.json`	Notification, logbook and Grafana credentials, in plain text
`conf/alerts.json`, `conf/alert_log.json`	The alert rules, and the history of what they raised	Yes
`calib/<board>.cal`	Per-channel energy calibration	Yes
`app.db`	Run metadata, users	Yes
`data/run<N>/`	The run itself — see below	Yes

A run directory contains:

File	Written	Contents
`*.caendat`	during	The data. Split across files when the size limit is reached.
`<board>.json`	at start	The board configurations this run was taken with.
`current.txt`	during	Beam current, one row per sample.
`stats.csv`	during	The monitored Graphite metrics, sampled on the rate cadence.
`roi.json`	at stop	Every defined ROI, its bounds, and its final counts.
`metadata.json`	at stop	The run’s FAIR metadata record.

The point of the per-run copies is that a run is self-describing: months later, the ROOT file, the settings it was taken with, the charge it collected and the regions it was analysed with are all in one folder.

7. Monitoring

Three different things, often confused:

	Answers	Lives
Spectra and waveforms	“What is this detector doing right now?”	In memory, from CaenDAQ. Not stored.
Rates (event, pile-up, lost, saturation)	“Is the DAQ keeping up?”	Published to Graphite every cadence tick; also `stats.csv`.
Slow control (beam current, accelerator values)	“What are the beam conditions?”	Graphite, plus `current.txt` and `stats.csv`.

Spectra are live and disposable — they reset when a run starts. Rates and slow control are archived in Graphite, which is what makes it possible to ask what the beam was doing three days ago. See Monitoring and alerts.

8. After the run

.caendat is a raw format. To analyse a run, convert it to ROOT with RUReader, which WebDAQ installs and drives from the conversion tab: pick a run, pick the options, convert. Point it at the run directory and a run split across several files converts into one ROOT file.

scripts/check_run_integrity.py verifies a run’s files without converting them — useful when a run was interrupted and you want to know how much of it is good.

9. Interfaces


Web	The main interface. Any browser on the network; see the README for reaching it from another PC.
Mobile	A React Native app for run status, spectra and logbook entries from a phone. It lives in its own repository and is not part of this checkout.
REST API	Everything the browser does is an HTTP call the server serves, so scripting a run is a matter of calling the same endpoints. JWT-authenticated. See the Server Architecture Guide.

The frontend has no privileged path into the DAQ — it is one API client among several, which is why a browser crash cannot take a run with it.