Container-Based Installation

This guide covers installing and running METADA using containers, which provides a consistent environment across different platforms.

Singularity/Apptainer Installation

Prerequisites

Install Singularity/Apptainer on your system:

Linux:

# For Ubuntu/Debian
sudo apt update
sudo apt install singularity-container

# For RHEL/CentOS
sudo yum install singularity

macOS:

# Using Homebrew
brew install singularity

Windows:

Use Windows Subsystem for Linux (WSL2) and follow Linux instructions.

Using Pre-built Container

Pull the METADA container:

singularity pull metada.sif oras://ghcr.io/metada/metada:latest

Run the container:

# Linux/WSL with GPU support
singularity shell --nv metada.sif

# macOS or without GPU
singularity shell metada.sif

Building Custom Container

Create a definition file metada.def:

metada.def

Bootstrap: docker
From: nvidia/cuda:12.1.0-devel-ubuntu22.04

%labels
    Author "METADA Team"
    Version 0.1.0
    Description "Modern Data Assimilation System for Earth Sciences"

%environment
    export LC_ALL=C
    export PATH=/usr/local/cuda/bin:$PATH
    export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH

%post
    # Update system and install basic dependencies
    apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \
        build-essential \
        cmake \
        ninja-build \
        git \
        python3 \
        python3-pip \
        python3-dev \
        python3-numpy \
        libboost-all-dev \
        libeigen3-dev \
        gfortran \
        wget \
        libgtest-dev \
        clang-format \
        lcov \
        libyaml-cpp-dev \
        nlohmann-json3-dev \
        && apt-get clean \
        && rm -rf /var/lib/apt/lists/*

    # Install Python dependencies
    pip3 install --no-cache-dir \
        numpy \
        scipy \
        matplotlib \
        sphinx \
        sphinx_rtd_theme

    # Create workspace directory
    mkdir -p /opt/metada
    cd /opt/metada

    # Set up build environment variables
    export CUDA_HOME=/usr/local/cuda
    export PATH=${CUDA_HOME}/bin:${PATH}
    export LD_LIBRARY_PATH=${CUDA_HOME}/lib64:${LD_LIBRARY_PATH}

%runscript
    echo "METADA Singularity Container"
    echo "CUDA Version: $(nvcc --version | grep release | awk '{print $5}')"
    echo "Python Version: $(python3 --version)"
    exec "$@"

%test
    python3 -c "import numpy; print('NumPy:', numpy.__version__)"
    nvcc --version
    cmake --version

Build the container:

sudo singularity build metada.sif metada.def

Docker Installation

Prerequisites

Install Docker:
For GPU support:
- Linux: Install NVIDIA Container Toolkit
- Windows: Enable WSL2 backend in Docker Desktop

Using Pre-built Container

Pull the METADA container:

docker pull ghcr.io/metada/metada:latest

Run the container:

# With GPU support
docker run --gpus all -it ghcr.io/metada/metada:latest

# Without GPU
docker run -it ghcr.io/metada/metada:latest

Building Custom Container

Create a Dockerfile:

Dockerfile

# Start from NVIDIA CUDA base image
FROM nvidia/cuda:12.1.0-devel-ubuntu22.04

# Set environment variables
ENV DEBIAN_FRONTEND=noninteractive
ENV PYTHONUNBUFFERED=1
ENV PATH=/usr/local/cuda/bin:$PATH
ENV LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH

# Install system dependencies
RUN apt-get update && apt-get install -y \
    build-essential \
    cmake \
    ninja-build \
    git \
    python3 \
    python3-pip \
    python3-dev \
    python3-numpy \
    libboost-all-dev \
    libeigen3-dev \
    gfortran \
    doxygen \
    graphviz \
    wget \
    libgtest-dev \
    clang-format \
    lcov \
    libyaml-cpp-dev \
    nlohmann-json3-dev \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*

# Install Python packages
RUN pip3 install --no-cache-dir \
    numpy \
    scipy \
    matplotlib \
    sphinx \
    sphinx_rtd_theme

# Create workspace directory
WORKDIR /opt/metada

# Copy CMake configuration
COPY CMakeLists.txt .
COPY cmake ./cmake

# Copy source code
COPY src ./src
COPY tests ./tests
COPY docs ./docs

# Build the project
RUN cmake -S . -B build \
    && cmake --build build \
    && ctest --test-dir build --output-on-failure

# Set default command
CMD ["/bin/bash"]

Build the container:
```
docker build -t metada:custom .
```

Development Workflow

VS Code Integration

Install required VS Code extensions:
- Remote Development
- Docker (for Docker workflow)

Configure container development:

For Docker:

# Create a .devcontainer directory
mkdir -p .devcontainer

# Create a basic devcontainer configuration
cat > .devcontainer/devcontainer.json << 'EOF'
{
    "name": "METADA Development",
    "image": "ghcr.io/metada/metada:latest",
    "runArgs": ["--gpus", "all"],
    "customizations": {
        "vscode": {
            "extensions": [
                "ms-vscode.cpptools",
                "ms-vscode.cmake-tools",
                "twxs.cmake"
            ]
        }
    },
    "remoteUser": "vscode"
}
EOF

Mount source code and build:

Singularity:

singularity shell --nv -B /path/to/source:/opt/metada metada.sif

Docker:

docker run --gpus all -v /path/to/source:/opt/metada -it metada:latest

Building and Testing

Inside the container:

cd /opt/metada
cmake -S . -B build -G Ninja \
  -DCMAKE_BUILD_TYPE=Debug \
  -DCMAKE_CXX_STANDARD=17
cmake --build build -j$(nproc)
ctest --test-dir build --output-on-failure

Troubleshooting

Common Issues

GPU not available: - Verify NVIDIA drivers are installed - Check container GPU support flags (--nv for Singularity, --gpus all for Docker) - Ensure NVIDIA Container Toolkit is installed (Docker on Linux)
Permission denied: - Check bind mount permissions - Verify user permissions in container - Use appropriate flags for volume mounts
Build failures: - Verify all dependencies are included in container - Check compiler compatibility - Ensure sufficient resources (memory/disk space)
Performance issues: - Monitor GPU utilization - Check memory allocation - Verify container resource limits