The last statistics events in each slice are now correctly imported.
Previously, in the surprising edge case where two statistic periods are
reported with the same end-of-slice timestamp, they were both being duplicated
when marking the end of the slice.
Also, there were scattered instances where the final statistics in a slice
weren't being imported at all or were being imported incorrectly. These are
now fixed as well.
This was bumped once already after the previous release version, but
bumping it again now will make sure that all beta testers are working
from data based on the current state of the PRS1 loader.
This now correctly shows gaps in therapy and statistics when the
mask is off. It also corrects the initial statistics for some sessions
to 1 second later, when the initial mask-on slice begins 1 second
after the session starts.
Weird zero-length PB and LL events are now dropped on import, since
they wouldn't get drawn anyway and seem to be peculiar artifacts.
Because summaries used to be parsed incorrectly, there was a bunch of ugly
logic that was designed to infer the session start & end times from other data.
Now that the summary parsers work, that can all go away. The only change to
imported data is that sessions ending with the mask off are now longer, to
reflect that extra time before the machine was turned off.
This now correctly shows gaps in therapy and statistics when the
mask is off. It currently only works for F3V3, since it has a
separate event chunk per slice. Other machines, which have only
a single large event chunk, will need to track the slices as
they import events.
This also highlights the issue with the last statistics in a
slice: only one data point is imported indicating the start of
a measured value, and so it appears that the data terminates then
instead of at the end of the slice (seconds to minutes later).
This will let us create "duplicate" statistical events at the end of
a session (and eventually slice) to reflect the end time of each
measurement channel.
Occasionally waveforms in a DreamStation session can be split into
multiple files. This behavior resulted in a report of missing waveform
data, and upon investigation was found 15 times out of 10,000 sample
sessions.
It looks like this happens when the machine begins writing the waveform
file before realizing that it will hit its 500-file-per-directory limit
for the remaining session files, at which point it appears to write
the rest of the waveform data along with the summary and event files
in the next directory.
The previous commit added better testing support and warning messages
for when this is encountered. This commit fixes the issue, and so
the warning is no longer necessary.
Also warn when waveform files are being skipped on import due to this
kind of splitting.
The chunks YAML now emits all unique chunks found in files with the
same session ID.
Note that a single file can contain multiple chunks covering multiple
sessions. These will all be saved in the YAML file corresponding to the
original file's name, rather than the session ID encoded in any chunk.
This slight discrepancy is intentional, since the chunk YAML is meant
to test the parsers, to verify that they correctly decode a specific
input file. When importing data into a session, we use the actual
session ID specified by each chunk. Thus the session YAML files will
be derived from the proper chunks, regardless of their original
containing file. (Well, except for waveforms, but they don't appear
to have more than one session ID per file.)
F3V3 events are formatted as waveforms (rather than a series of timestamped
events), with one chunk per mask-on slice. The previous assumption of only
a single event chunk per session meant that all events after the first slice
were getting dropped.
BiPAP AVAPS 30 (System One 60 Series) now works much better, although the data
doesn't yet show any gaps where the mask is off. Fixing that requires changes
throughout the code and not just in the PRS1 loader.
Also, LL calculation seems wrong. Perhaps the automatic calculation of
unintentional leak from total leak doesn't work for AVAPS, throwing off
automatic LL calculation? (F3V3 reports neither unintentional leakage nor
large leak events.)
This commit adds a mapping from PRS1 events to the OSCAR channels that
should receive the data, along with the necessary supporting code to
allow for generic handling of PRS1 events as a result.
The F3V6 importer has minimal changes to partially use the new
plumbing, enough to make sure that it works and causes no change in
the imported data.
F3V3 is unusual. Its events are recorded at fixed intervals like a waveform.
Its summary is recorded with absolute timestamps. Slices are each given their
own chunks in the event file.
This last oddity isn't yet handled.
While examining this, a few other importing issues have been cleaned up:
PRS1DataChunk::ParseEvents no longer needs the mode passed in as an argument,
F5V012 importers should quietly eat the newly parsed pressure
adjustment events until we can import both those and average pressure
statistics, and F5V012 importers should handle real VS events and treat
VS statistics like the other importers do.
Based on the F0V23 loop and F0V6 switch statement, along with manual review of
several sample files to determine record length.
Split ParseHumidifierSettingF0V4 out from ParseSettingsF0V4.
Also fix an issue with initializing the model names, since QObject::tr
won't work at global initialization time. And series detection needs
the untranslated names anyway.
Also clean up brick detection.
The official names don't yet appear anywhere, since there's a question
of how to juggle manufacturer, series, and model name in the various
places they're (inconsistently) displayed.
Series is also used to pick the machine icon.
Pressure settings are now properly being found and decoded, but there are lots
of unknown fields to figure out.
It turns out it uses the same humidifier setting encoding as F0V6, and the first
several slices seem to be the same. But pressure encodings are different, with
a gain of 0.125 instead of 0.1, presumably to allow for a maximum pressure of
30 cmH2O.
Lots of debug messages and unknown values, but the structure seems right.
This is largely based on a combination of the mainblock parsing for fileVersion 3
as found in ParseSummary() and the switch statements of ParseSummaryF0V6,
based on our understanding of slices from F0V23. The slice types here
come from sample 200X files.
Also add some debug messages to FV3 parsing. It's clear the current
approach is wrong. This looks a lot like the slices seen earlier,
since hbdata values appear more than once in a given file.
Also turn off summary YAML since the next bit of work will focus on
parsing.
Remarkably, this resulted in nearly no changes to the test output.
Only one session with 0 duration had its start/end times reset to 0,
since it will no longer get imported.
Only PRS1Import::ParseSummaryF0V23 has disappeared, being merged into
ImportSummary. The only change it needed to subsume the functionality of
all other variants was the addition of one extra setting.
The rest of the now-unused variants will be removed next.
