Data Structure¶
DOLfYN data objects are built on (subclasses of) pyDictH5 data objects, which are themselves subclasses of Python dictionaries. As such, these objects support all of the same basic functionality of dictionaries (e.g., indexing, iterating, etc.), with additional functionality that is designed to streamline the process of analyzing and working with data. Most notably, these data objects provide:
a summarized view of the data structure when in interactive mode:
>>> dat <ADV data object> . 1.05 hours (started: Jun 12, 2012 12:00) . INST-frame . (120530 pings @ 32Hz) *------------ | mpltime : <time_array; (120530,); float64> | vel : <array; (3, 120530); float32> + config : + DATA GROUP + env : + DATA GROUP + orient : + DATA GROUP + props : + DATA GROUP + signal : + DATA GROUP + sys : + DATA GROUP
attribute-style syntax (
dat.vel
is equivalent todat['vel']
)direct access of items within sub-groups (
dat['env.temp']
works, and'env.temp' in dat
evaluates toTrue
)
See the pyDictH5 README for additional details.
Data Description and Units¶
DOLfYN generally uses the *MKS* system, with most angles in degrees.
Name | Units | Group | Description/Notes |
---|---|---|---|
mpltime | matplotlib date format | — | The time data |
vel | m/s | — | The velocity vector data |
range | m | — | The distance from the ADP head |
depth_m | m | — | The depth of the ADP |
c_sound | m/s | env | The speed of sound used in velocity calculations. |
press / pressure | dbar? | env | pressure measured by the instrument |
temp | degrees Celsius | env | The temperature measured by the instrument. |
accel | m/s2 | orient | The vector acceleraton of the instrument |
angrt | rad/s | orient | The angular rotation rate of the instrument |
mag | ??? | orient | The magnetometer data. |
velraw | m/s | — | The velocity without motion-correction. |
velrot | m/s | orient | The rotational motion velocity (computed from angrt). |
velacc | m/s | orient | The translational motion velocity (computed from accel). |
acclow | m/s2 | orient | The low-pass filtered acceleration signal. |
heading | degrees | orient | The instrument heading* (clockwise from North) |
pitch | degrees | orient | The instrument pitch* |
roll | degrees | orient | The instrument roll* |
orientmat | — | orient | The instrument’s orientation matrix in the earth reference frame |
amp | ??? | signal | The acoustic signal return amplitude |
corr | percent | signal | correlation between the send and receive pings. |
ambig_vel | m/s | sys | The ambiguity velocity* |
batt_V / batt | voltage | sys | The battery voltage |
ensemble | — | sys | The ensemble counter |
error | — | sys | The instrument error code* |
status | — | sys | The instrument status code* |
temp_clock | degrees C? | sys | The instruments clock temperature |
temp_mag | degrees C? | sys | The instruments magnetometer temperature |
temp_press | degrees C? | sys | The instruments pressure sensor temperature |
tke_vec | m2/s2 | — | Variance of velocity components (0: uu, 1: vv, 2: ww) |
stress | m2/s2 | — | The Reynolds stress array (0: uv, 1: uw, 2: vw) |
sigma_Uh | m/s | — | Standard deviation of horizontal velocity magnitude |
epsilon | m2/s3 | — | Turbulence dissipation rate |
Spec | <variables units>2/(rad/s) | — | A data group containing spectra of variables above. |
omega | rad/s | Spec | Radial frequency. |
Data Shortcuts (properties)¶
In addition to the data items listed above, DOLfYN data objects also
contain shortcuts to tools and other variables that can be obtained
from simple operations of its data items. These attributes aren’t
listed in the view of the data shown above. Instead, to see the
variables that are available as shortcuts for a particular data
object, take a look at the dat.shortcuts
property (new in DOLfYN
0.10.1).
Name | units | Description/Notes |
---|---|---|
shape | — | the shape of dat.u (see below) |
n_time | — | the length of dat['mpltime'] |
u | m/s | dat['vel'][0] |
v | m/s | dat['vel'][1] |
w | m/s | dat['vel'][2] |
U | m/s | the horizontal velocity as a complex quantity (dat.u + 1j * dat.v) |
U_mag | m/s | the magnitude of the horizontal velocity |
U_angle | radians | the direction of the horizontal velocity (counter-clockwise from East, X, or streamwise direction, depending on coordinate system) |
subset | — | a tool for indexing a subset of the data object |
tauij | m2/s2 | the Reynold’s stress tensor |
tke | m2/s2 | the turbulent kinetic energy (half the sum of the data in ‘tke_vec’) |
upup_ | m2/s2 | dat['tke_vec'][0] |
vpvp_ | m2/s2 | dat['tke_vec'][1] |
wpwp_ | m2/s2 | dat['tke_vec'][2] |
upvp_ | m2/s2 | dat['stress'][0] |
upwp_ | m2/s2 | dat['stress'][1] |
vpwp_ | m2/s2 | dat['stress'][2] |
Ecoh | m2/s2 | Coherent turbulent kinetic energy (root-sum-square of Reynold’s stresses) |
I | — | Ratio of sigma_Uh to horizontal velocity magnitude |
Itke | — | Ratio of sqrt(2*tke) to horizontal velocity magnitude |
Important Note: The items listed in Table 3 are not stored in the data object but are provided as attributes (shortcuts) to DOLfYN data objects.
User Meta-Data (dat.props
)¶
The props
data-group of DOLfYN data objects is a place for
user-specified meta-data and DOLfYN-specific implementation data. The
most common variables found here are described in Table 2.
Name | Description/Notes |
---|---|
fs* | This is the sample rate of the instrument [Hz] |
coord_sys* | The coordinate system of the data object. When a data object is rotated to a new coordinate system using the dat.rotate2() method, the value of dat.props['coord_sys'] is updated to reflect the final coordinate system. Valid values are: beam, inst, earth, and principal. For further details on these coordinate systems see the Rotations and Coordinate Systems section. |
rotate_vars* | The variables in the data object that should be rotated when rotating the data object. |
declination* | The magnetic declination where the measurements were made (in degrees that magnetic North is right of True North). Set this value using dat.set_declination(<value>) . |
declination_in_heading* | A boolean specifying whether the heading includes the declination. If this is True, then the earth coordinate system is True (i.e., v is velocity toward True North). |
declination_in_orientmat* | A boolean specifying whether the orient.orientmat includes the declination. If this is True, then the earth coordinate system is True (i.e., v is velocity toward True North). |
principal_heading | The heading of the +u direction for the ‘principal’ coordinate system [degrees clockwise from north]. |
has imu* | A boolean indicating whether the instrument has an inertial motion sensor/unit. |
inst_make* | The manufacturer name. |
inst_model* | The instrument model. |
DutyCycle_NBurst* | The number of pings in a burst. |
DutyCycle_NCycle* | The time – in number of pings – before the next burst starts. (this may be incorrect for some instrument types, please report issues) |
body2head_vec | The vector from the center of the ADV body reference frame to the center of the head’s reference frame, in coordinates of the body reference frame. For ADPs this is always zero because the two coordinate systems are centered at the same place. |
body2head_rotmat | The rotation matrix that rotates vectors from the instrument body reference-frame to the head reference-frame. For ADPs this is always the identity matrix. Valid values are: 3-by-3 valid rotation matrices, or: 'eye' , 'identity' , or 1 all of which specify that it is the identity matrix. This is typically used for cable-head ADVs where the ADV head is not oriented the same as the ADV body. |
motion accel_filtfreq Hz | The filter-frequency for the computing translational motion from the acceleration signal of an IMU. This is only used for motion correction. This high-pass filter is applied prior to integrating acceleration. This value is only used if when accel_filtfreq is not explicitly specified when motion-correcting |
motion vel_filtfreq Hz | The filter-frequency for the computing translational motion from the acceleration signal of an IMU. This is only used for motion correction. This high-pass filter is applied after integrating acceleration. This value is only used if when vel_filtfreq is not explicitly specified when motion-correcting. |
lonlat† | The location of the measurements in decimal degrees. Longitude is positive west of the prime-meridian, and latitude is positive North of the equator. |
*: These entries are set by DOLfYN, and should not - in general - be set or changed by the user.
†: These entries are not used or set by DOLfYN, but they are useful measurement meta-data and are listed here to assist in standardizing the location and format of this information.
Specify meta-data in a JSON file¶
The values in dat.props
can also be set in a json file,
<data_filename>.userdata.json
, containing a single json-object. For example, the contents of these files should
look something like:
{"body2head_rotmat": "identity",
"body2head_vec": [-1.0, 0.5, 0.2],
"motion accel_filtfreq Hz": 0.03,
"declination": 8.28,
"lonlat": [-105.2283, 39.9402]
}
Prior to reading a binary data file my_data.VEC
, you can
create a my_data.userdata.json
file. Then when you do
dolfyn.read('my_data.VEC')
, DOLfYN will read the contents of
my_data.userdata.json
and include that information in the
dat.props
attribute of the returned data object. This
feature is provided so that meta-data can live alongside your
binary data files.