Welcome to EnviroDIY, a community for do-it-yourself environmental science and monitoring. EnviroDIY is part of Home › Forums › Mayfly Data Logger › Mayfly not writing to SD card = possibly a libraries issue?
- This topic has 8 replies, 5 voices, and was last updated 2021-11-02 at 12:12 PM by
dan@wachusett.
-
AuthorPosts
-
-
2021-09-17 at 2:38 PM #15902
Dear Mayfly community,
Recently we’ve expanded our terrestrial Mayfly network to include 7 new stations in a research forest in Vermont. We installed 2 stations in the fall and data was logged just fine.
We set everything up identically, using the same code and the same libraries. Set the RTC, addressed all the sensors. No problem. Checked the data through the serial monitor – great. But for whatever reason, the boards do not log data to the SD card.
The code is identical, and the libraries on the laptop used to upload the code is identical. The set up is nearly identical. We even took the Mayflys back to the lab and tested everything, and they wrote to the SD card just fine. So, we brought them back out and – nothing was written.
At this point – we’re stuck. We tried formatting the SD card differently (which is the only thing that we think could possibly be the difference) and that didn’t work. For some reason, when the Mayfly is allowed to run code, sleep, and awaken and collect data, it is not writing into a text file anymore.
I’m not sure if this could possibly be a conflicting library issue – since the libraries are updated from time to time. What else could be the root of a failure to write to the SD card from a board perspective? We bought these mayflys as part of a kit, so there is the capacity to use the vertical SD card, but we are not using that. Otherwise, we are quite stumped and any help would be appreciated!
Dave
-
2021-09-17 at 4:13 PM #15903
If they wrote to the SD card in the lab but not the field could it be a power supply issue?
-
2021-09-17 at 4:16 PM #15904
That’s a good thought – the only thing is that we have the identical setup nearby that we installed in November with the exact same battery and solar panel, and those function with no problem. We don’t believe there is a difference in the physical infrastructure that we can tell – so the only difference could be what libraries we installed?
-
-
2021-09-18 at 9:48 PM #15905
An idea, equipment failure isolation. (assuming you’ve got the board in a lab location) Possible combinations of the three variable 1) switch uSD and 2) switch Mayfly 3) add separate USB power supply via USB cable. Does the Mayfly always fail writing to the uSD, even with known working uSD. That could be that the uSD slot has a failure (effectively Mayfly bad). I get a white/red SD card so that I give each SD card a number on the white portion so I can track them.
-
2021-09-28 at 4:56 PM #15933
A small update for everyone:
We were able to allow the Mayfly to operate by updating the libraries in our code. We are using some core IDE libraries as well as some EnviroDIY libraries. My sense is that there might be a conflict between the recent update to the SD library, and perhaps a pin interrupt library from EnviroDIY, but I’m not sure.
One particular clue is that for the mayflys that are not operating when connected to the battery, their RTC clock is also not operational. So, when we pull the SD card, not only is there no file written, but when we check what is written to the serial, the RTC is still set to the date we initially set it in August.
I’m not sure if Shannon or Sara have any insights into possible issues with library updates and SD card functionality. It writes to the serial when connected to a computer, but just won’t initialize the txt file to write to the external file on the SD.
Dave
-
2021-09-28 at 5:32 PM #15935
I haven’t had any issues with any loggers not recording data recently or clocks not running, including boards I just programmed this week. But I use stable libraries from several months ago and haven’t updated them recently. Are you using all the libraries that are available in our suggested .zip file, found here: https://github.com/EnviroDIY/Libraries and are you using one of our sample logger sketches, or one that you’re written or modified?
-
2021-10-06 at 2:51 PM #16046
Shannon,
We had been using the EnviroDIY Libraries as well as libraries contained within the IDE itself. Our code uses sections of the sample logger sketch, but have been built out a bit to combine possible analog sensors as well as communicate with Xbee.
I’d be happy to share with you – it’s not super long and you’ll see that it is pretty simple.
Another thought I had dealt with power supplies. I know the Mayfly has a 12V step-down in it. We have each of our stations connected to a charge controller linked to a solar panel and a deep-cycle marine battery so that our stations can operate even in the winter time. I wonder if power is entering at a voltage higher than 12V (low 14s) if this might cause an issue.
A clue really does seem to be the RTC. When it is not operational and doesn’t retain the date/time, the Mayfly will not write. But we have brand new batteries in the RTC so that’s why we thought it could be a conflict in the libraries somehow?
Dave
-
-
2021-09-28 at 5:55 PM #15936
Sara captures all the libs specified as part of the release procedure https://github.com/EnviroDIY/ModularSensors/releases/tag/v0.30.0 look for ModularSensors_Dependencies_v0.30.0.zip
I’ve not had any issue with 0.30.0.. The tests are heavily utilizing the SD for reliable web delivery, storing them to uSD, before a POST.
I do a build from scratch including pulling all the libs when I do a release.
https://github.com/neilh10/ModularSensors/releases/tag/v0.30.0.release1_210831Building 210831_1234
Dependency Graph|– <EnviroDIY_ModularSensors> 0.30.0+sha.bc3dd0e
| |– <EnableInterrupt> 1.1.0
| |– <SdFat> 2.0.7
| | |– <SPI> 1.0
| |– <TinyGSM> 0.11.4 -
2021-11-02 at 12:12 PM #16092
I am experiencing a similar issue that may be related, although without the RTC issue…
- Built with Modular sensors v0.30.0
- Data is saved to SD card when connected to computer via cable. RTC time is correct (or close enough)
- If I disconnect the board from the computer it fails to wake up or log any data to the SD card
- Battery voltage is sufficient (3.88 V, tested with multimeter)
- One of my boards was operating fine using a previous sketch and prior version of modular sensor dependencies, so I don’t think its a hardware issue. I am experiencing the exact same problem on 6 boards.
I’ve included my sketch below (UUIDs and Tokens have been scrubbed). I have seen sketch examples (DRWI_LTE.ino) where the wake pin is set to 31, whereas I have it set to A7. Perhaps this is the issue? If anyone has any thoughts or suggestions for troubleshooting please let me know.
Thanks,
Dan
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367/*****************************************************************************Mayfly_WACH-MD01.inoWritten By: Sara Damiano (sdamiano@stroudcenter.org)Development Environment: Arduino IDEHardware Platform: EnviroDIY Mayfly Arduino DataloggerSoftware License: BSD-3.Copyright (c) 2017, Stroud Water Research Center (SWRC)and the EnviroDIY Development TeamThis example sketch is written for ModularSensors library version 0.23.16This sketch is an example of logging data to an SD card and sending the data toboth the EnviroDIY data portal as should be used by groups involved withThe William Penn Foundation's Delaware River Watershed InitiativeDISCLAIMER:THIS CODE IS PROVIDED "AS IS" - NO WARRANTY IS GIVEN.* Modified by Dan Crocker - October 2021*****************************************************************************/// ==========================================================================// Defines for the Arduino IDE// In PlatformIO, set these build flags in your platformio.ini// ==========================================================================#ifndef TINY_GSM_RX_BUFFER#define TINY_GSM_RX_BUFFER 64#endif#ifndef TINY_GSM_YIELD_MS#define TINY_GSM_YIELD_MS 2#endif// ==========================================================================// Include the base required libraries// ==========================================================================#include <Arduino.h> // The base Arduino library#include <EnableInterrupt.h> // for external and pin change interrupts#include <LoggerBase.h> // The modular sensors library// ==========================================================================// Data Logger Settings// ==========================================================================// The name of this fileconst char* sketchName = "Mayfly_WACH-MD01.ino";// Logger ID, also becomes the prefix for the name of the data file on SD cardconst char* LoggerID = "WACH-MD01";// How frequently (in minutes) to log dataconst uint8_t loggingInterval = 15;// Your logger's timezone.const int8_t timeZone = 0; // UTC// NOTE: Daylight savings time will not be applied! Please use standard time!// ==========================================================================// Primary Arduino-Based Board and Processor// ==========================================================================#include <sensors/ProcessorStats.h>const long serialBaud = 115200; // Baud rate for the primary serial port for debuggingconst int8_t greenLED = 8; // MCU pin for the green LED (-1 if not applicable)const int8_t redLED = 9; // MCU pin for the red LED (-1 if not applicable)const int8_t buttonPin = 21; // MCU pin for a button to use to enter debugging mode (-1 if not applicable)const int8_t wakePin = A7; // MCU interrupt/alarm pin to wake from sleep// Set the wake pin to -1 if you do not want the main processor to sleep.// In a SAMD system where you are using the built-in rtc, set wakePin to 1const int8_t sdCardPwrPin = -1; // MCU SD card power pin (-1 if not applicable)const int8_t sdCardSSPin = 12; // MCU SD card chip select/slave select pin (must be given!)const int8_t sensorPowerPin = 22; // MCU pin controlling main sensor power (-1 if not applicable)// Create the main processor chip "sensor" - for general metadataconst char* mcuBoardVersion = "v0.5b";ProcessorStats mcuBoard(mcuBoardVersion);// Create battery voltage variable pointers for the processorVariable* mcuBoardBatt = new ProcessorStats_Battery(&mcuBoard, "12345678-abcd-1234-ef00-1234567890ab");/** End [processor_sensor] */// ==========================================================================// Wifi/Cellular Modem Options// ==========================================================================/** Start [xbee_cell_transparent] */// For any Digi Cellular XBee's// NOTE: The u-blox based Digi XBee's (3G global and LTE-M global)// are more stable used in bypass mode (below)// The Telit based Digi XBees (LTE Cat1) can only use this mode.#include <modems/DigiXBeeCellularTransparent.h>// Create a reference to the serial port for the modemHardwareSerial& modemSerial = Serial1; // Use hardware serial if possibleconst int32_t modemBaud = 9600; // All XBee's use 9600 by default// Modem Pins - Describe the physical pin connection of your modem to your board// NOTE: Use -1 for pins that do not applyconst int8_t modemVccPin = -2; // MCU pin controlling modem powerconst int8_t modemStatusPin = 19; // MCU pin used to read modem statusconst bool useCTSforStatus = false; // Flag to use the modem CTS pin for statusconst int8_t modemResetPin = 20; // MCU pin connected to modem reset pinconst int8_t modemSleepRqPin = 23; // MCU pin for modem sleep/wake requestconst int8_t modemLEDPin = redLED; // MCU pin connected an LED to show modem// status (-1 if unconnected)// Network connection informationconst char* apn = "vzwinternet"; // The APN for the gprs connectionDigiXBeeCellularTransparent modemXBCT(&modemSerial, modemVccPin, modemStatusPin,useCTSforStatus, modemResetPin,modemSleepRqPin, apn);// Create an extra reference to the modem by a generic nameDigiXBeeCellularTransparent modem = modemXBCT;// Create RSSI and signal strength variable pointers for the modemVariable* modemRSSI = new Modem_RSSI(&modem, "12345678-abcd-1234-ef00-1234567890ab");Variable* modemSignalPct = new Modem_SignalPercent(&modem, "12345678-abcd-1234-ef00-1234567890ab");// ==========================================================================// Maxim DS3231 RTC (Real Time Clock)// ==========================================================================#include <sensors/MaximDS3231.h>// Create a DS3231 sensor objectMaximDS3231 ds3231(1);// Create a temperature variable pointer for the DS3231Variable* ds3231Temp = new MaximDS3231_Temp(&ds3231, "12345678-abcd-1234-ef00-1234567890ab");// Deleted Campbell Scientific Sensor - Not Applicable here// ==========================================================================// Meter Hydros 21 Conductivity, Temperature, and Depth Sensor// ==========================================================================/** Start [hydros21] */#include <sensors/MeterHydros21.h>const char* hydrosSDI12address = "1"; // The SDI-12 Address of the Hydros 21const uint8_t hydrosNumberReadings = 6; // The number of readings to averageconst int8_t SDI12Power = sensorPowerPin; // Power pin (-1 if unconnected)const int8_t SDI12Data = 7; // The SDI12 data pin// Create a Meter Hydros 21 sensor objectMeterHydros21 hydros(*hydrosSDI12address, SDI12Power, SDI12Data,hydrosNumberReadings);// Create specific conductance depth and temperature variable pointers for the Hydros21 sensorVariable* Hydros21Cond =new MeterHydros21_Cond(&hydros, "12345678-abcd-1234-ef00-1234567890ab");Variable* Hydros21Depth =new MeterHydros21_Depth(&hydros, "12345678-abcd-1234-ef00-1234567890ab");Variable* Hydros21Temp =new MeterHydros21_Temp(&hydros, "12345678-abcd-1234-ef00-1234567890ab");/** End [hydros21] */// ==========================================================================// Creating the Variable Array[s] and Filling with Variable Objects// ==========================================================================Variable* variableList[] = {Hydros21Cond,Hydros21Depth,Hydros21Temp,mcuBoardBatt,ds3231Temp,modemRSSI,modemSignalPct};// *** CAUTION --- CAUTION --- CAUTION --- CAUTION --- CAUTION ***// Check the order of your variables in the variable list!!!// Be VERY certain that they match the order of your UUID's!// Rearrange the variables in the variable list if necessary to match!// *** CAUTION --- CAUTION --- CAUTION --- CAUTION --- CAUTION ***const char* REGISTRATION_TOKEN = "12345678-abcd-1234-ef00-1234567890ab"; // Device registration tokenconst char* SAMPLING_FEATURE = "12345678-abcd-1234-ef00-1234567890ab"; // Sampling feature UUID// Count up the number of pointers in the arrayint variableCount = sizeof(variableList) / sizeof(variableList[0]);// Create the VariableArray objectVariableArray varArray(variableCount, variableList);// Put only the particularly interesting variables into a second array// NOTE: We can the same variables into multiple arraysVariable* variableList_toGo[] = {Hydros21Cond,Hydros21Depth,Hydros21Temp};// Count up the number of pointers in the arrayint variableCount_toGo = sizeof(variableList_toGo) / sizeof(variableList_toGo[0]);// Create the VariableArray objectVariableArray arrayToGo(variableCount_toGo, variableList_toGo);/** End [variable_arrays] */// ==========================================================================// The Logger Object[s]// ==========================================================================// Create a new logger instanceLogger dataLogger(LoggerID, loggingInterval, &varArray);// Create "another" logger for the variables to go out over the internetLogger loggerToGo(LoggerID, loggingInterval, &arrayToGo);// ==========================================================================// A Publisher to WikiWatershed// ==========================================================================// Device registration and sampling feature information can be obtained after// registration at http://data.WikiWatershed.orgconst char* registrationToken = REGISTRATION_TOKEN; // Device registration tokenconst char* samplingFeature = SAMPLING_FEATURE; // Sampling feature UUID// Create a data publisher for the EnviroDIY/WikiWatershed POST endpoint#include <publishers/EnviroDIYPublisher.h>EnviroDIYPublisher EnviroDIYPOST(loggerToGo, &modem.gsmClient, registrationToken, samplingFeature);// ==========================================================================// Working Functions// ==========================================================================// Flashes the LED's on the primary boardvoid greenredflash(uint8_t numFlash = 4, uint8_t rate = 75){for (uint8_t i = 0; i < numFlash; i++) {digitalWrite(greenLED, HIGH);digitalWrite(redLED, LOW);delay(rate);digitalWrite(greenLED, LOW);digitalWrite(redLED, HIGH);delay(rate);}digitalWrite(redLED, LOW);}// Read's the battery voltage// NOTE: This will actually return the battery level from the previous update!float getBatteryVoltage(){if (mcuBoard.sensorValues[0] == -9999) mcuBoard.update();return mcuBoard.sensorValues[0];}// ==========================================================================// Main setup function// ==========================================================================void setup(){// Start the primary serial connectionSerial.begin(serialBaud);// Print a start-up note to the first serial portSerial.print(F("Now running "));Serial.print(sketchName);Serial.print(F(" on Logger "));Serial.println(LoggerID);Serial.println();// Serial.print(F("Using ModularSensors Library version "));// Serial.println(MODULAR_SENSORS_VERSION);// if (String(MODULAR_SENSORS_VERSION) != String(libraryVersion))// Serial.println(F(// "WARNING: THIS EXAMPLE WAS WRITTEN FOR A DIFFERENT VERSION OF MODULAR SENSORS!!"));// Start the serial connection with the modemmodemSerial.begin(modemBaud);// Set up pins for the LED'spinMode(greenLED, OUTPUT);digitalWrite(greenLED, LOW);pinMode(redLED, OUTPUT);digitalWrite(redLED, LOW);// Blink the LEDs to show the board is on and starting upgreenredflash();// Set the timezones for the logger/data and the RTC// Logging in the given time zoneLogger::setLoggerTimeZone(timeZone);// It is STRONGLY RECOMMENDED that you set the RTC to be in UTC (UTC+0)Logger::setRTCTimeZone(0);// Attach the modem and information pins to the loggerdataLogger.attachModem(modem);loggerToGo.attachModem(modem);modem.setModemLED(modemLEDPin);dataLogger.setLoggerPins(wakePin, sdCardSSPin, sdCardPwrPin, buttonPin, greenLED);// Begin the loggerdataLogger.begin();// Note: Please change these battery voltages to match your battery// Check that the battery is OK before powering the modemif (getBatteryVoltage() > 3.55 || !dataLogger.isRTCSane()){modem.modemPowerUp();modem.wake();modem.setup();// Synchronize the RTC with NISTSerial.println(F("Attempting to connect to the internet and synchronize RTC with NIST"));if (modem.connectInternet(120000L)){dataLogger.setRTClock(modem.getNISTTime());}else{Serial.println(F("Could not connect to internet for clock sync."));}}// Set up the sensors, except at lowest battery levelif (getBatteryVoltage() > 3.4){Serial.println(F("Setting up sensors..."));varArray.setupSensors();}// Power down the modemmodem.modemSleepPowerDown();// Create the log file, adding the default header to it// Do this last so we have the best chance of getting the time correct and// all sensor names correct// Writing to the SD card can be power intensive, so if we're skipping// the sensor setup we'll skip this too.if (getBatteryVoltage() > 3.4){dataLogger.turnOnSDcard(true); // true = wait for card to settle after power updataLogger.createLogFile(true); // true = write a new headerdataLogger.turnOffSDcard(true); // true = wait for internal housekeeping after write}// Call the processor sleepSerial.println(F("Putting processor to sleep\n"));dataLogger.systemSleep();}// ==========================================================================// Main loop function// ==========================================================================// Use this short loop for simple data logging and sendingvoid loop(){// Note: Please change these battery voltages to match your battery// At very low battery, just go back to sleepif (getBatteryVoltage() < 3.4){dataLogger.systemSleep();}// At moderate voltage, log data but don't send it over the modemelse if (getBatteryVoltage() < 3.55){dataLogger.logData();}// If the battery is good, send the data to the worldelse{dataLogger.logData();//loggerToGo.publishDataToRemotes(); // disable until SIM cards are installed}}
-
-
AuthorPosts
- You must be logged in to reply to this topic.
