[Sara Damiano]

Scott Ensign, formerly of Planktos Instruments, is now at Stroud. (He’s @ensign.)

I don’t know how “unusual” trajectory data is, but, at least in long term stream monitoring, it’s definitely far less common than fixed station monitoring. That type of data is also harder to manage, visualize, or conceptualize. I’d guess if others don’t support trajectory data it’s for those same reasons. Even including just site-specific data our feature wish list is far longer than we have the resources to ever complete.

What were you hoping to accomplish by uploading your data to MonitorMW? If your goal is to have your data easily sharable and searchable, I’d recommend using HydroShare (https://www.hydroshare.org/landingPage/). Hydroshare supports uploading ODM2 databases and netCDF multi-dimensional data. HydroShare is also a DOI provider for permanently tagging specific datasets. If you want to manage your own data, you might look into CZ Manager (https://github.com/ODM2/CZ-Manager), which is based on ODM2 (like MonitorMW) but gives much, much deeper database control. You could also look into the tools provided by groups like NCAR (https://ncar.ucar.edu/) – I don’t have any personal familiarity with any of them though. I don’t actually know of any spiffy online data viewers, to send you to, but I haven’t ever looked either.

I hope one of those is more helpful! I’m sorry MonitorMW isn’t ready for your data yet!

[Sara Damiano]

@ensign has done some work with a moving/drifting sensor stations: https://www.fondriest.com/news/hydrosphere-drifter-brings-lagrangian-sampling-freshwater.htm

The way Monitor My Watershed’s data portal is set up right now is very “site-centric” – the web pages are just not set up to accept moving sensor data like you would generate. The database model which the website is built on can manage trajectory data, but the portal isn’t exposing that portion of the database. To put the data in right now you could pretend it’s all from a single station and then call latitude and longitude two of the your measured variables, but that’s definitely a cludgey way to do it.

For how the trajectory results can work in ODM2, see: https://github.com/ODM2/ODM2/blob/master/doc/ODM2Docs/ext_results_measurementframeworks.md and https://github.com/ODM2/ODM2/blob/master/doc/ODM2Docs/ext_results_trajectory.md

[Sara Damiano]

Well it’s certainly not intended that less than 100% of the data makes it online, if that’s what your asking.

Beyond that I can’t really help without a lot more information. So, here goes:

Do you know where in the chain the data is being lost? Are you using a Mayfly? What code are you running? If ModularSensors, what version of both ModularSensors itself and of TinyGSM. What firmware version do you have on the XBee3? What firmware version do you have on the SARA R410 inside of the XBee3? Are you using the the XBee3 in transparent or bypass mode?

1. Is 100% of the data on your SD card? If so, the sensors are working and the logging portion of the code on your board is working fine; move on to 2. If not:
1a. Are you recording the “ProcessorStats_SampleNumber?” If so, is this number being reset after a gap? If so, something is causing your board to freeze until the watch-dog timer goes off. That could be an electrical/wiring issue or a code issue.
1b. If you’re not recording sample number, How long is the gap between points when data is dropped? If the gaps are always either 15 or 20 mintues long, that’s probably the watch dog. It bites after 15 minutes, but you may miss the first new point after it bites leaving a 20 minute gap.
1c. If you think it’s the watchdog:
1ci. Are you using any I2C sensors and are disconnecting their power between sampling. If the I2C bus is held low, it will cause the low level I2C/Wire library to hang on the next attempt to communicate with any I2C device (like the RTC).
1cii. There is also an issue with the XBee3 in bypass mode that sometimes causes hangs. I know it happens. I can’t figure out why. You can read about it on GitHub. I’ve only seen this sort of hang once every few days or so, though, not 2.4% of the time.

2. Is the issue with both wifi and cellular or just with the cellular? If both, skip to 4.

3. Can you see your data use by individual sessions for your cellular plan (or wifi connection)? If you are using a hologram data plan, you can see this on their dashboard.
3a. When looking at the data sessions on hologram, do you see a session at every single 5 minute interval. If not, you either don’t have good enough service or there’s some other issue with the modem seeing and registering to the internet.
3b. If you see a session at every interval, do you see actual data being used on that session or are they 0 byte sessions? When running the XBee3 in transparent mode it will sometimes continue to report that it is unable to register to the network even though on hologram’s side the registration has been successful. At other times (again, when in transparent mode) the XBee3 fails to properly packet the data and actually send it out. Both of these things happens more on older version of the XBee3 firmware, but I’ve still seen it on the current version. I’ve tried a number of things to try to help the connection, but unfortunately this is on Digi to fix. I have had many, many fewer issues with empty sessions when running in Bypass mode, at the trade off of a know bug causing rare hangs in bypass.

4. Finally, if you’re see the data on your SD card, you’re seeing sessions that consume at every single five minute interval, and your still not seeing data on the data portal, it could be something wrong either with the portal receiving the data or displaying it.
4a. Is the data missing no matter how you attempt to view it? (ie, sparklines, pop-up table, downloaded csv from the data portal, plots in WebTSA, csv download from WebTSA) If it’s missing everywhere it’s *probably* not being processed and put into the “main” databased correctly.
4b. If you can see data on any place in the database, but just not all of them, there are known communication issues between the “main” database and the higher performance “daughter” databases that produce the various types of plots and other ways of viewing the data. The data isn’t lost and hopefully these issues will be fixed soon, but the only solution in this case is to wait.

[Sara Damiano]

https://github.com/ODM2/ODM2DataSharingPortal/issues/443

[Sara Damiano]

I’m stumped. I have no idea why it’s failing. Your format is correct, your UUID is correct and in the right place… I have no idea. I’ll report it as a bug.

[Sara Damiano]

Delete them.

[Sara Damiano]

If you don’t have a CTD attached, you should remove all the lines for the CTD. If you don’t remove them, your logger will waste time trying to communicate with a sensor that doesn’t exist and you’ll be wasting bandwidth sending empty values to the data portal.

[Sara Damiano]

I’m sorry the directions for that example are confusing – it was originally written for people directly working with Stroud and being assigned loggers by us. If you’re making your own logger outside of Stroud, you don’t have to worry about Stroud assigning your logger a number.

[Sara Damiano]

If you’re not working with Stroud, you get to assign your logger any ID you want it to have. We often use the Mayfly serial number or a numeric count of it being the nth logger we’ve deployed. You might also want to put your site name in the logger id, since it will end up as your file name. You could also call it “Bob” if you’d prefer. 🙂

[Sara Damiano]

The Digi documents don’t tell you what commands are exchanged between the u-blox chip – I don’t if the response the XBee3 gives to the “ATAI” command is based on CREG, CEREG, or some combination of those and any other commands that Digi found to be useful – and I doubt Digi is going to tell us. I suspect that it is a combination of commands.

If you’re interested in using bypass mode, you should reference the AT command manual from u-blox.

[Author]

Posts