.. _directory_structure:

===========================================
Directory Structure and File Layout
===========================================

Overview
--------

Oceanarray organises files in a **mooring-first** layout: the top-level
directory for each data type (raw or processed) contains one subdirectory
per mooring, and instrument-type subdirectories appear beneath that.  This
contrasts with an older **instrument-first** layout (used with the legacy
``--basedir`` flag) where the raw directory grouped files by instrument type
across all moorings.

The mooring-first layout makes it easier to work on a single mooring at a
time: all raw files, processed files, YAML configuration, and reports for
one mooring live under a common directory that can be copied, backed up, or
inspected without touching other moorings.

.. note::

   If you are coming from the ``--basedir`` layout, see :doc:`migration`.

----

Raw directory layout
--------------------

The raw directory (supplied with ``--raw-dir``) holds the instrument data
files downloaded from the instruments at sea.  No processing is done in this
directory; it is treated as read-only by ``oceanarray``.

.. code-block:: text

   {raw_dir}/                          ← supplied with --raw-dir (cruise level)
   └── {mooring_name}/                 ← one directory per mooring
       ├── microcat/                   ← instrument: value from the YAML clamp entry
       │   ├── 5367_recovery.asc
       │   └── 26261_recovery.asc
       ├── aquadopp/
       │   ├── A400115_dsG3.aqd
       │   └── A400115_dsG3.hdr       ← header file for Nortek instruments
       └── ADCP/
           └── 12345_raw.000

The subdirectory name under the mooring directory must exactly match the
``instrument`` field in the YAML configuration (e.g. ``microcat``,
``aquadopp``, ``ADCP``).  The ``filename`` field in each YAML instrument
entry is relative to that instrument subdirectory.

.. note::

   You may also store instrument setup and capture files alongside the data
   files in the instrument subdirectory.  These are not read by
   ``oceanarray`` but are useful for provenance:

   - **SeaBird instruments**: the ``.cap`` capture file and ``.xmlcon``
     configuration file downloaded with the data.
   - **RBR instruments**: the ``.log`` file generated by Ruskin at download.
   - **Nortek Aquadopps**: the deployment configuration output (e.g.
     ``A400115_dsG3.prf`` or the Deployment Details summary).
   - **RDI ADCPs**: the command-log text file recording the instrument settings
     used during the deployment.

----

Processed directory layout
--------------------------

The processed directory (``--proc-dir``) receives all outputs from
``oceanarray process`` (including its ``stack`` and ``grid`` stages) and
``oceanarray report``.

.. code-block:: text

   {proc_dir}/                             ← supplied with --proc-dir
   └── {mooring_name}/
       ├── {mooring_name}.mooring.yaml     ← YAML configuration lives here
       ├── microcat/
       │   ├── {mooring}_{serial}_stage1.nc
       │   ├── {mooring}_{serial}_stage2.nc
       │   └── {mooring}_{serial}_stage3.nc
       ├── aquadopp/
       │   ├── {mooring}_{serial}_stage1.nc
       │   ├── {mooring}_{serial}_stage2.nc
       │   └── {mooring}_{serial}_stage3.nc
       ├── ADCP/
       │   └── ...
       ├── {mooring}_stack.nc              ← all instruments on a common time grid
       ├── {mooring}_grid.nc               ← vertically interpolated onto a pressure grid
       ├── processing_logs/
       │   └── {mooring}_{timestamp}_{stage}.mooring.log
       └── report/
           ├── {mooring}_report.html           ← mooring summary report
           ├── {mooring}_stack_report.html
           ├── {mooring}_grid_report.html
           └── instrument/
               └── {mooring}_{serial}_report.html

----

The YAML configuration file
----------------------------

The mooring YAML file must be placed inside the processed directory for
that mooring::

   {proc_dir}/{mooring_name}/{mooring_name}.mooring.yaml

The ``{mooring_name}`` component is the value of the ``name`` field at the
top of the YAML file (e.g. ``name: dsG3_1_2026``).  The directory name,
the YAML filename, and the ``name`` field must all match exactly.

Oceanarray looks for the YAML in this location when you supply
``--proc-dir``.  You do not need to place a copy in the raw directory.

If you do not yet have a processed directory for a mooring, create one
manually before running any ``oceanarray`` command::

   mkdir -p /path/to/proc/dsG3_1_2026

Then copy or write the YAML into that directory.  See :doc:`yaml_configuration`
for a full description of the YAML fields and a minimal working example.

----

File naming conventions
-----------------------

See :doc:`file_naming_conventions` for the full reference, including both raw
instrument file names (mooring recovery vs calibration-dip conventions) and
processed output file names (stage NC files, stack, grid, reports, logs).