Metadata-Version: 2.1
Name: npc_lims
Version: 0.1.161
Summary: Tools to fetch and update paths, metadata and state for Mindscope Neuropixels sessions, in the cloud.
Author-email: Arjun Sridhar <arjun.sridhar@alleninstitute.org>, Ben Hardcastle <ben.hardcastle@alleninstitute.org>
License: MIT
Project-URL: Repository, https://github.com/AllenInstitute/npc_lims
Project-URL: Issues, https://github.com/AllenInstitute/npc_lims/issues
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: POSIX :: Linux
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: npc_session>=0.1.30
Requires-Dist: redis>=4.1.4
Requires-Dist: aind-codeocean-api>=0.4.0
Requires-Dist: pydbhub-bjh>=0.0.8
Requires-Dist: pyyaml>=6.0.1
Requires-Dist: pyopenssl>=23.2.0
Requires-Dist: openpyxl>=3.1.2
Requires-Dist: packaging>=23.2
Requires-Dist: types-pyYAML>=6.0.12.12
Requires-Dist: types-requests>=2.31.0.6
Requires-Dist: npc-io>=0.1.24

# npc_lims

**n**euro**p**ixels **c**loud **l**ab **i**nformation **m**anagement **s**ystem
Tools to fetch and update paths, metadata and state for Mindscope Neuropixels sessions, in the cloud.

[![PyPI](https://img.shields.io/pypi/v/npc-lims.svg?label=PyPI&color=blue)](https://pypi.org/project/npc-lims/)
[![Python version](https://img.shields.io/pypi/pyversions/npc-lims)](https://pypi.org/project/npc-lims/)

[![Coverage](https://img.shields.io/codecov/c/github/alleninstitute/npc_lims?logo=codecov)](https://app.codecov.io/github/AllenInstitute/npc_lims)
[![CI/CD](https://img.shields.io/github/actions/workflow/status/alleninstitute/npc_lims/publish.yml?label=CI/CD&logo=github)](https://github.com/alleninstitute/npc_lims/actions/workflows/publish.yml)
[![GitHub
issues](https://img.shields.io/github/issues/alleninstitute/npc_lims?logo=github)](https://github.com/alleninstitute/npc_lims/issues)

## quickstart

- make a new Python >=3.9 virtual environment with conda or venv (lighter option, since this package does not require pandas, numpy etc.):
  ```bash
  python -m venv .venv
  ```
- activate the virtual environment:
  - Windows
  ```cmd
  .venv\scripts\activate
  ```
  - Unix
  ```bash
  source .venv/bin/activate.sh
  ```
- install the package:
  ```bash
  python -m pip install npc_lims
  ```
- setup credentials
  - required environment variables:
    - AWS S3
      - `AWS_DEFAULT_REGION`
      - `AWS_ACCESS_KEY_ID`
      - `AWS_SECRET_ACCESS_KEY`
      - to find and read files on S3
      - must have read access on relevant aind buckets
      - can be in a standard `~/.aws` location, as used by AWS CLI or boto3
    - CodeOcean API
      - `CODE_OCEAN_API_TOKEN`
      - `CODE_OCEAN_DOMAIN`
      - to find processed data in "data assets" via the Codeocean API
      - generated in CodeOcean:
        - right click on `Account` (bottom left, person icon)
        - click `User Secrets` - these are secrets than can be made available as environment variables in CodeOcean capsules
        - go to `Access Tokens` and click `Generate new token` - this is for programatically querying CodeOcean's databases
          - in `Token Name` enter `Codeocean API (read)` and check `read` on capsules and datasets
          - a token will be generated: click copy (storing it in a password manager, if you use one)
        - head back to `User Secrets` where we'll paste it into a new secret via `Add secret > API credentials` - in `description` enter `Codeocean API (read)` - in `API key` enter `CODE_OCEAN_API_KEY` - in `API secret` paste the copied secret from before (should start with `cop_`...)
          `CODE_OCEAN_DOMAIN` is the codeocean https address, up to and including `.org`
  - environment variables can also be specified in a file named `.env` in the current working directory
    - example: https://www.dotenv.org/docs/security/env.html
    - be very careful that this file does not get pushed to public locations, e.g. github
      - if using git, add it to a `.gitignore` file in your project's root directory:
      ```gitignore
      .env*
      ```
- now in Python we can find sessions that are available to work with:

  ```python
      >>> import npc_lims;

  # get a sequence of `SessionInfo` dataclass instances, one per session:
      >>> tracked_sessions: tuple[npc_lims.SessionInfo, ...] = npc_lims.get_session_info()

  # each `SessionInfo` instance has minimal metadata about its session:
      >>> tracked_sessions[0]                 # doctest: +SKIP
      npc_lims.SessionInfo(id='626791_2022-08-15', subject=626791, date='2022-08-15', idx=0, project='DRPilotSession', is_ephys=True, is_sync=True, allen_path=PosixUPath('//allen/programs/mindscope/workgroups/dynamicrouting/PilotEphys/Task 2 pilot/DRpilot_626791_20220815'))
          >>> tracked_sessions[0].is_ephys        # doctest: +SKIP
      False

  # currently, we're only tracking behavior and ephys sessions that use variants of https://github.com/samgale/DynamicRoutingTask/blob/main/TaskControl.py:
      >>> all(s.date.year >= 2022 for s in tracked_sessions)
      True

  ```

- "tracked sessions" are discovered via 3 routes:
  - https://github.com/AllenInstitute/npc_lims/blob/main/tracked_sessions.yaml
  - `\\allen\programs\mindscope\workgroups\dynamicrouting\DynamicRoutingTask\DynamicRoutingTraining.xlsx`
  - `\\allen\programs\mindscope\workgroups\dynamicrouting\DynamicRoutingTask\DynamicRoutingTrainingNSB.xlsx`
