Dataset: CTD_hydrography
Deployment: EN278

Procedure for processing primary CTD data:

The following is an outline of the methods used to process the "primary" CTD data from the 1995 Broad scale surveys.

1. Upon completion of each broad scale cruise, preliminary processing of the MK5 data was carried out:
   1. "first differencing" (called from within CTDPOST) of the Raw cast files was completed. This program flags any data where the difference between sequential scans exceed some preset limit.
   2. "smart editor" used to interpolate over any flagged values from "first differencing"
   3. ENDCAST XXX (a dos batch routine where xxx=cast#) runs MK5PROC.EXE which creates pressure averaged, pressure centered 1 db files (.PRE files). These routines (pressure averaging and pressure centering) were developed by Bob Millard at WHOI and were modified for the Mk5. "ENDCAST" also updates a cruise header file, and backs up the data to floppy)
   4. CTDPLOT (visual basic program by David Mountain) was run at each station to plot out the cast profile and to visually inspect the data.
   5. MK5BOT.EXE This routine (again, developed by Bob Millard) extracts and averages 30 data scans around the data scan number (as identified in the cast .BTL file) at which bottles were fired during a cast and appends to a cruise bottle file. These files were reviewed after each cruise to ensure that each btl fired had a corresponding observation record in the btl file. If we forgot to record a btl fired (forgot to press [CTRL] F3), we used an observation from the .PRE files.
2. Salinity samples (btm) collected on each cruise were analyzed on a guildline autosal
   • tstslt.bas was used to compare the cruise btl file with the autosal output file
   • mk5xxxx.m (where xxxx=4 digit cruise id) was used to calculate the mean conductivity offset
   1. any single comparison that was greater than -0.1 or +0.1 was ignored.
   2. the mean offset and standard deviation (std) were calculated
   3. any single comparison that was greater than +/- 2 std's was discarded as an outlier
   4. the mean and std were re-calculated
   5. The offset series for each cruise were inspected to insure that a time dependant trend was not evident and that using a mean offset was appropriate.
       
3. The mean conductivity offsets were applied to each cruise data set using MK5final.bas. MK5cru.fin was read in as input containing the necessary information for cruise id, directory, and most importantly conductivity offset. The output was pressure averaged, pressure centerred .PRS files.
   • During EN261 and ALB9506, seabird data was used to supplement for primary hydro data when there were problems with the MK5 (or winch). Seabird CTD data were processed to pressure averaged, 1 db files using the manufacturer's software DATCNV, ALIGNCTD, BINAVG, DERIVE, and ASCIIOUT. When Seabird data were used as "primary" data, the casts were numbered >100. The same steps outlined in 2a)--2e) were followed in the quality controlling of the seabird salinity data except that the comparison was done with salinity (rather than conductivity). The seabird cast data have also been processed to NODC formatted files and are available in this format in a NEFSC anonymous ftp account ftp: ftp/pub/hydro.
      
4. Final data checking was accomplished with a matlab program called dchkXX.m (where xx is the cruise code) that did the following:
   1. read in the cruise .h3 file (output of ENDCRUIS.FOR) and calculated the distance and time between consecutive casts and wrote the results to headchk.xx (where xx is the cruise code). Any speed that was >= 10 knots was verified with the original cruise logs and if found in error, was corrected.
   2. read in the cast files and went through a series of allowable "range checking" of T, S, Sigma-t, and fluorescence. Profiles of these 4 parameters are plotted to the screen for visual inspection.
      

                              RANGES
                         0 <--- T ---> 25
                        30 <--- S ---> 37
                         0 <--- Fl ---> 5
                     -0.05 <--- ST---> 0.1 
      
      

      The above flagged values were written out to datachk.xx

** note: in most instances, the positive sigma-t differences were O.K. and occurred when stratification had developed or when the MK5 sampled through strong gradients like the shelf / slope front. However, there were cases of "hysteresis" (sp?) when the MK5 or seabird sampled through sharp t/s gradients. In these instances, I had to delete the records.

** On a number of occasions, there were negative fluorescence values for the sfc observation. (These were also observed in the raw data...A consultant with Sea-tech said that this is probably "noise" associated with its equilibrating when just powered up). For these casts, I looked in the raw data files, and chose a substitute fluorescence value.

The output listing of the flagged observations from "datachk.m" are kept on file (and can be made available). Any record deletions are noted.

The following is an outline of the methods used to process the
"primary" CTD data from the 1995 Broad scale surveys.
<ol>
<li>Upon completion of each broad scale cruise, preliminary
processing of the MK5 data was carried out:

<ol>

<li><b>"first differencing"</b> (called from within CTDPOST) of
the Raw cast files was completed. This program flags
any data where the difference between sequential
scans exceed some preset limit. </li> <p>

</p><li><b>"smart editor"</b> used to interpolate over any flagged
values from "first differencing"</li><p>

</p><li><b>ENDCAST XXX</b> (a dos batch routine where xxx=cast#)
runs MK5PROC.EXE which creates pressure averaged,
pressure centered 1 db files (.PRE files). These
routines (pressure averaging and pressure centering)
were developed by Bob Millard at WHOI and were
modified for the Mk5. "ENDCAST" also updates a
cruise header file, and backs up the data to floppy) </li><p>

</p><li><b>CTDPLOT</b> (visual basic program by David Mountain)
was run at each station to plot out the cast
profile and to visually inspect the data.</li><p>

</p><li><b>MK5BOT.EXE</b> This routine (again, developed by Bob
Millard) extracts and averages 30 data scans around
the data scan number (as identified in the cast .BTL
file) at which bottles were fired during a cast and
appends to a cruise bottle file. These files were
reviewed after each cruise to ensure that each btl
fired had a corresponding observation record in the
btl file. If we forgot to record a btl fired
(forgot to press [CTRL] F3), we used an observation
from the .PRE files.</li><p>
</p></ol>

</li><li>Salinity samples (btm) collected on each cruise were analyzed
on a guildline autosal
<ul>
<li><b>tstslt.bas</b> was used to compare the cruise btl
file with the autosal output file</li>

<li><b>mk5xxxx.m</b> (where xxxx=4 digit cruise id) was
used to calculate the mean conductivity offset</li>
</ul>
<p>
</p><ol>
<li>any single comparison that was greater than
-0.1 or +0.1 was ignored.</li>
<li>the mean offset and standard deviation (std) were
calculated</li>

<li>any single comparison that was greater than
+/- 2 std's was discarded as an outlier</li>
<li>the mean and std were re-calculated</li>
<li>The offset series for each cruise were inspected to
insure that a time dependant trend was not evident
and that using a mean offset was appropriate.</li><p>
</p></ol> <p>

</p></li><li>The mean conductivity offsets were applied to each cruise data
set using <b>MK5final.bas</b>. <b>MK5cru.fin</b> was read in as input
containing the necessary information for cruise id, directory,
and most importantly conductivity offset. The output was
pressure averaged, pressure centerred .PRS files.
<p></p><ul>

<li>During EN261 and ALB9506, seabird data was used to supplement
for primary hydro data when there were problems with the MK5 (or
winch). Seabird CTD data were processed to pressure averaged, 1 db
files using the manufacturer's software <b>DATCNV, ALIGNCTD, BINAVG,
DERIVE,</b> and <b>ASCIIOUT</b>. When Seabird data were used as "primary"
data, the casts were numbered >100. The same steps outlined in
2a)--2e) were followed in the quality controlling of the seabird
salinity data except that the comparison was done with salinity
(rather than conductivity). The seabird cast data have also been
processed to NODC formatted files and are available in this format
in a NEFSC anonymous ftp account ftp: ftp/pub/hydro.</li>
</ul><p>

</p></li><li>Final data checking was accomplished with a matlab program
called <b>dchkXX.m</b> (where xx is the cruise code) that did the
following:
<p>

</p><ol>
<li>read in the cruise .h3 file (output of <b>ENDCRUIS.FOR</b>) and
calculated the distance and time between consecutive casts
and wrote the results to headchk.xx (where xx is the cruise
code). Any speed that was >= 10 knots was verified
with the original cruise logs and if found in error,
was corrected.</li><p></p>

<li>read in the cast files and went through a series of
allowable "range checking" of T, S, Sigma-t, and
fluorescence. Profiles of these 4 parameters are plotted to
the screen for visual inspection.<p>
</p><pre>
RANGES
0 <--- T ---> 25
30 <--- S ---> 37
0 <--- Fl ---> 5
-0.05 <--- ST---> 0.1

</pre>
The above flagged values were written out to <b>datachk.xx</b> <p>
</p></li></ol></li>
</ol>
<p>
** note: in most instances, the positive sigma-t differences were
O.K. and occurred when stratification had developed or when the MK5
sampled through strong gradients like the shelf / slope front.
However, there were cases of "hysteresis" (sp?) when the MK5 or
seabird sampled through sharp t/s gradients. In these instances,
I had to delete the records.</p><p>
</p><p>
** On a number of occasions, there were negative fluorescence
values for the sfc observation. (These were also observed in the
raw data...A consultant with Sea-tech said that this is probably
"noise" associated with its equilibrating when just powered up).
For these casts, I looked in the raw data files, and chose a
substitute fluorescence value.</p>