Previously it was reporting spurious information when the humidifier was
disconnected. Now it only reports settings details when the humidifier is
connected (the only time they're valid.)
It also now reports the humidification mode (fixed, adaptive, or heated
tube), and the heated tube temperature when appropriate.
The last statistics events in each slice are now correctly imported.
Previously, in the surprising edge case where two statistic periods are
reported with the same end-of-slice timestamp, they were both being duplicated
when marking the end of the slice.
Also, there were scattered instances where the final statistics in a slice
weren't being imported at all or were being imported incorrectly. These are
now fixed as well.
If there are no mask-on slices of nonzero duration, then there's not any
meaningful event or waveform data for the session. There is occasionally some
fragmentary data, but it's always less than 1 second.
When such fragmentary waveform data is present, it only contains 1-3 nonzero
samples, corresponding to 0.2s - 0.6s of data, which suggests that the
mask-on slice was really that long rather than precisely 0. As a result,
it appears that the timestamps of the mask-on/mask-off slices are just the
current value of the machine's internal clock, which only has 1-second
resolution.
But rather than embark on herculean efforts to derive a sub-second slice
duration from (only occasionally present) event or waveform data, we just
treat the session as having no detailed data.
Because summaries used to be parsed incorrectly, there was a bunch of ugly
logic that was designed to infer the session start & end times from other data.
Now that the summary parsers work, that can all go away. The only change to
imported data is that sessions ending with the mask off are now longer, to
reflect that extra time before the machine was turned off.
Also clean up unused PRS1 channels and fix a regression from
9a25647c that broke unintentional leak calculation for CPAP/APAP.
Also add PRS1PressureAverageEvent to F0V4 list of supported events.
Right now this results in slightly confusing graphs, since sometimes
the IPAP/EPAP data for a machine is primarily reported by its settings
events, and for other machines by its statistics events. Also, the
"average" pressure on F0 looks like it might be effectivley EPAP in
single-pressure mode rather than the true average in bi-level mode.
Once we decide on the best presentation, we can either update the importer
or the display to show the most helpful channels for a given session.
Also warn when waveform files are being skipped on import due to this
kind of splitting.
The chunks YAML now emits all unique chunks found in files with the
same session ID.
Note that a single file can contain multiple chunks covering multiple
sessions. These will all be saved in the YAML file corresponding to the
original file's name, rather than the session ID encoded in any chunk.
This slight discrepancy is intentional, since the chunk YAML is meant
to test the parsers, to verify that they correctly decode a specific
input file. When importing data into a session, we use the actual
session ID specified by each chunk. Thus the session YAML files will
be derived from the proper chunks, regardless of their original
containing file. (Well, except for waveforms, but they don't appear
to have more than one session ID per file.)
F3V3 is unusual. Its events are recorded at fixed intervals like a waveform.
Its summary is recorded with absolute timestamps. Slices are each given their
own chunks in the event file.
This last oddity isn't yet handled.
While examining this, a few other importing issues have been cleaned up:
PRS1DataChunk::ParseEvents no longer needs the mode passed in as an argument,
F5V012 importers should quietly eat the newly parsed pressure
adjustment events until we can import both those and average pressure
statistics, and F5V012 importers should handle real VS events and treat
VS statistics like the other importers do.
It also turns out that every sample machine that claims to be F0V3 in its
property file contains only F0V2 data, so there appears to be nothing
additional to test, but now there are more warnings in case we ever do
actually encounter F0V3 data.
Also warn if we ever encounter oximetry.
Still trying to figure out timestamp event.
Other summary parsers will need review, since there was an initial
filter that was dropping sessions that began with event 5 or 6,
now commented out.
This one's slice 8 is shorter than anything else we've seen before!
Also add support for a new slice type and noted possible values
for other data, including a possible lead on the CPAP mode.
The new slice type appears when humidifier settings have changed during
the session, and includes a timestamp. The session time now matches
reports.
Otherwise just comments to indicate other values seen in the sample data,
including a lead on humidifier encoding.
Also add some debug messages to FV3 parsing. It's clear the current
approach is wrong. This looks a lot like the slices seen earlier,
since hbdata values appear more than once in a given file.
Also turn off summary YAML since the next bit of work will focus on
parsing.
It turns out the session end time was intentionally not being set in ParseSummary,
probably due to its unreliability. This may be revisited once things are more
stable, but for now the old behavior is retained.
Remarkably, this resulted in nearly no changes to the test output.
Only one session with 0 duration had its start/end times reset to 0,
since it will no longer get imported.
Also move an extra unsupported check out of PRSImport::ImportSummary
into CreateMachineFromProperties, where it should intervene before
reaching PRS1Import. Leave a warning debug message in its place.
Each input file's chunks get emitted into a single output YAML file. As parsing
gets separated from conversion/import, this will allow for testing and
examination of the parsed input files before they are transformed into
sessions.
This fix also seems like it cleans up some of the weird session IDs. It appears
that session IDs were sticking around after each machine, adding those
IDs or sessions to subsequent machines in that run of the test. Since the
test was being run piecemeal while under development, this produced
different behavior when being run with different segments of the data.
This has already exposed many limitations, and possibly some memory trampling. Before this will work as
a true regression test, we'll need to address both of those, so that this produces reliable, reproducible
output.
