Dataset: Mysid movies - Damariscotta Estuary - Summer 2023

Name: Movies of mysid aggregations in Damariscotta Estuary, Maine obtained during the summer of 2023
Published: 2024-04-25
Keywords: Mysid ecology, In situ behavioral observations, Stereoimaging, oceans

Cite This Dataset

DOI:10.26008/1912/bco-dmo.925893.1

Spatial Extent: N:43.93419 E:-69.57958 S:43.93419 W:-69.57958

Damariscotta Estuary, in the vicinity of the Darling Marine Center, Walpole, Maine, USA

Temporal Extent: 2023-07-26 - 2023-08-04

Project:

Collaborative Research: Characterizing benthic mysid ecology and animal-fluid interactions in response to background flow, food, and light conditions (Mysid ecology)

Principal Investigator:

Aditya Nayak (Florida Atlantic University, FAU)

Scientist:

Sumit Dass (Florida Atlantic University, FAU)

Student:

Alexis Base (Florida Atlantic University, FAU)

BCO-DMO Data Manager:

Amber D. York (Woods Hole Oceanographic Institution, WHOI BCO-DMO)

Version:

Version Date:

2024-04-25

Restricted:

Yes

Release Date:

2025-08-03

Validated:

Yes

Current State:

Final no updates expected

Movies of mysid aggregations in Damariscotta Estuary, Maine obtained during the summer of 2023

Abstract:

Mysid shrimp are a group of zooplankton that are distributed in diverse aquatic environments – freshwater, marine and estuarine – in both shallow and deep waters throughout the world. Due to their ubiquitous presence, mysids form a critical link in marine food webs; however, much remains unknown about their ecology and interactions with their local surroundings. This study aims to conduct a comprehensive investigation of mysid behavior in their natural environment, including their interactions and distribution under varying local flow, food and light conditions. The survey will involve developing and deploying a novel in situ imaging system to study mysids in coastal Maine over the span of two summers. This dataset contains raw movies of mysid aggregations obtained during summer 2023.

Expand/Collapse All

Archival Copy

Version Date	Archive	Persistent Identifier (PID)	Date PID Issued
2024-04-25	Marine Biological Laboratory/Woods Hole Oceanographic Institution Library (MBLWHOI DLA)	10.26008/1912/bco-dmo.925893.1	2024-08-19

Description

This dataset contains image files and a metadata table in the "Data Files" section and additional "Supplemental Files" including processing code and a file inventory.

Methods & Sampling

For in situ mysid behavioral observations, we have developed and deployed a custom-built stereoimaging system capable of capturing synchronized high-quality images at a high frame rate (up to 21 fps). Our current setup features Teledyne Blackfly S GigE cameras with 5 MP resolution, IEEE1588 clock synchronization, and PoE support. Navitar lenses, with focal lengths of 12 mm or 25 mm, are attached via C-type lens mounts to ensure compatibility. Lighting is provided by a TrustFire DF50 scuba diving light with white and red LED modes. The stereo housing, designed in-house, comprises galvanized aluminum structures and acrylic end caps housing optical-grade glass lenses. It offers flexibility in inter-camera distance and angle. Custom-made underwater DragonFish Ethernet cables facilitate power and data transfer, enabling live feed monitoring, and synchronized camera frame storage, while also simplifying power management, thus extending deployment time.

The stereo housing and lighting system are affixed to an aluminum bar frame, which is lowered into the water using a crane. Initially, the frame is positioned at a shallow depth for calibration purposes. An 8x9 checkboard pattern with a 10 mm square size, printed on an aluminum composite panel, is submerged for calibration. Synchronized images are captured by both cameras as the checkerboard is moved to cover the common field of view and focused depth. Once calibration images are recorded, the camera rig is lowered to the desired recording depth. Live feed monitoring ensures proper lighting and clear visibility before recording synchronized images commences. After sufficient images are captured, recording is stopped, and the stored data undergoes quality and synchronization analysis. Finally, details such as the data recording location, folder identification, camera synchronization status, and the number of recorded images are documented in our field notes before proceeding to the next data recording session. Calibration is typically required only once at each location. File naming conventions: Maine2023_Pxx or Maine2023_Sxx as the naming protocol (for primary and secondary cameras, where xx represents the number of the dataset, e.g., P80, S80).

Problems/Issues

N/A

Data Processing Description

The original field data comprises 8-bit monochrome TIFF or BMP image files recorded synchronously by two cameras. The uploaded data consists of movies (MP4 video format) of the raw images, i.e., each movie is a compilation of all the raw images in a single dataset.

A Python script was used to process a sequence of image files from a specified folder (a single dataset) and generate an output video using the 'mp4v' codec. The video's frame rate matches that of the image recording rate. You can find the Python script code we used within a python notebook for this purpose within the supplemental file 'video_from_imgs.txt

BCO-DMO Processing

Mysid videos were bundled up into two zip packages with zip64 support using 7z. These two zip files were added to the "Data File" section (Mysid_movies_summer2023_Maine_P.zip, Mysid_movies_summer2023_Maine_S.zip).
* File inventory was created containing the imagename, filesize and checksums. Added columns to contain identifiers within the image name: image set id (primary or secondary folder), and Primary_or_Secondary indicating if it was the primary or secondary camera. (file_inventory.csv)

Sheet "Metadata" within file Maine_dataset_report_5.xlsx was imported into the BCO-DMO data system. With additional added columns with the associated primary and secondary movie name. Metadata table added as a supplemental file.
* Date converted to ISO 8601 format, ISO timestamps with timezone added (UTC) in addition to local times.
* Lat lon converted to decimal degrees

* Column names adjusted to conform to BCO-DMO naming conventions designed to support broad re-use by a variety of research tools and scripting languages. [Only numbers, letters, and underscores. Can not start with a number]

Data Files

File
Metadata for Primary and Secondary Camera Movies filename: 925893_v1_mysid-movies.csv (Comma Separated Values (.csv), 8.99 KB) MD5:1df61a3815bb0660fda71280147f8629 Collection and camera metadata associated with mysid movies (primary and secondary cameras). See "Parameters" section for column information (descriptions, units).
Mysid movies - Summer 2023 Maine - Primary Camera filename: Mysid_movies_summer2023_Maine_P.zip (ZIP Archive (ZIP), 27.34 GB) MD5:250abf79d1da588dbe81aa72974400db Zip package containing 39 videos (MP4 format) of the raw images from the primary camera. Each movie is a compilation of all the raw images in a single dataset (e.g. P80). See "Data processing" section for more details. File naming conventions: Maine_2023_Pxx_FPS_yy or Maine_2023_Sxx_FPS_yy as the naming protocol (for primary and secondary cameras, where xx represent the number of the dataset, e.g., P80, S80 and yy represents the frame rate at which the data is recorded).
Mysid movies - Summer 2023 Maine - Secondary Camera filename: Mysid_movies_summer2023_Maine_S.zip (ZIP Archive (ZIP), 27.11 GB) MD5:4767528598fa0d939cb97323abcb3774 Zip package containing 39 videos (MP4 format) of the raw images from the primary camera. Each movie is a compilation of all the raw images in a single dataset (e.g. P80). See "Data processing" section for more details. File naming conventions: Maine_2023_Pxx_FPS_yy or Maine_2023_Sxx_FPS_yy as the naming protocol (for primary and secondary cameras, where xx represent the number of the dataset, e.g., P80, S80 and yy represents the frame rate at which the data is recorded).

Supplemental Files

File

File inventory table
filename: file_inventory.csv

(Comma Separated Values (.csv), 9.74 KB)
MD5:05c82c9274ecdf1456ce7e7b1714c72e

Inventory table for each of the .mp4 files provided in this dataset.

Columns:
filename, video filename
filesize_bytes, filesize in bytes
md5sum, checksum (md5) that can be used to verify integrity of the file after transfers
within_zip_package, the name of the .zip package this file is within in the dataset

Python code for creating movie from Raw Images
filename: video_from_imgs.txt

(Plain Text, 2.90 KB)
MD5:bd45257f02cef522e6e28905cb9d4155

Python script which converts a sequence of images into a video file using OpenCV library. 
Amber's edit: Python code which converts a sequence of images into a video file using OpenCV library.  Code as used within a python notebook.

More Information about this dataset

Funding Sources

Funding Source	Award Number
NSF Division of Ocean Sciences	OCE-2138465

Deployments

None available yet

Instruments

Camera

Supplied Name: In-house custom-developed stereoimaging system for zooplankton observations: Teledyne Blackfly S GigE cameras

Supplied Description:

A custom-built stereoimaging system capable of capturing synchronized high-quality images at a high frame rate (up to 21 fps). Our current setup features Teledyne Blackfly S GigE cameras with 5 MP resolution, IEEE1588 clock synchronization, and PoE support. Navitar lenses, with focal lengths of 12 mm or 25 mm, are attached via C-type lens mounts to ensure compatibility. Lighting is provided by a TrustFire DF50 scuba diving light with white and red LED modes.

Instrument Type

Generic Name: Camera

Acronym: camera

Community Identifier: http://vocab.nerc.ac.uk/collection/L05/current/311/

Generic Description:

All types of photographic equipment including stills, video, film and digital systems.

Parameters

Date and time formats are described in python strftime() syntax.

Supplied Name	Supplied description	Supplied Units	Standard Name
Date_Local	The date on which the field data is recorded (time zone EST/EDT). Format: %Y-%m-%d	unitless	date_local
Start_Time_Local	The start time of field data recording. Format: %H:%M %p	unitless	time_local
End_Time_Local	The end time of field data recording. Format: %H:%M %p	unitless	time_local
Location	The location at which the data is recorded. The datas are recorded at two docks of the Darling Marine Center. DMC Pedestrian dock: Darling Marine Center Pedestrian dock; DMC Main dock: Darling Marine Center Mine Dock.	decimal degrees	site
Latitude	Latitude of the data recording location	decimal degrees	lat
Longitude	Longitude of the data recording location.	unitless	lon
Primary_Folder	Reference folder name of the camera which initiates the trigger for camera synchronization.	unitless	sample_descrip
Primary_Movie	Filename of the movie for the primary Primary_Folder images	unitless	sample_descrip
Secondary_Folder	Reference folder name of the camera which receives the trigger for camera synchronization.	unitless	sample_descrip
Secondary_Movie	Filename of the movie for the Secondary_Folder images	unitless	sample_descrip
Camera_FPS	Camera frame per second (FPS) set for the data recording. For all the recordings the same FPS is maintained for both cameras.	unitless	sample_descrip
Strobe_Light_Color	Strobe light color. The used strobe light is a battery-powered LED tourch which has four modes: Bright White/White/Red/UV.	unitless	sample_descrip
Image_Numbers	Image Numbers (Primary Camera/Secondary Camera ). Number of images captured by primary and secondary camera. Similar image numbers indicate that the cameras capture images in a synchronized manner.	unitless	sample_descrip
Camera_Orientation	Camera orientation description. Two different camera orientations are used: one parallel to the waterbed looking horizontally and the other vertical to the waterbed looking downwards.	unitless	sample_descrip
Start_DateTime_UTC	The start timestamp with timezone (UTC) of field data recording. Datetime format is ISO 8601. Format: %Y-%m-%dT%H:%MZ	unitless	ISO_DateTime_UTC
End_DateTime_UTC	The end timestamp with timezone (UTC) of field data recording. Datetime format is ISO 8601. Format: %Y-%m-%dT%H:%MZ	unitless	ISO_DateTime_UTC
Image_Storage_GB	Storage volume (Video). Camera captures 8-bit monochrome images saved in '.tiff' format. These values show the total hard drive space requied to save both the primary and secondary image folders.	Gigabytes (GB)	sample_descrip
Video_Storage_GB	Storage volume (Video). We employed a Python script to create a video file (.mp4 format) from a designated folder containing the raw image files (.tiff format). The column values indicate the cumulative hard drive space necessary to store the videos from both the primary and secondary folders.	Gigabytes (GB)	sample_descrip

Status message

Database

Contribute Data

Dataset: Mysid movies - Damariscotta Estuary - Summer 2023

Status message

Database

Contribute Data

Dataset: Mysid movies - Damariscotta Estuary - Summer 2023

View, Subset and Download Data

Data Access

Give Us Feedback