Welcome to EnviroDIY, a community for do-it-yourself environmental science and monitoring. EnviroDIY is part of WikiWatershed, an initiative of Stroud Water Research Center designed to help people advance knowledge and stewardship of fresh water.
New to EnviroDIY? Start here

Double_logger example modification

Home Forums Mayfly Data Logger Double_logger example modification

Viewing 11 reply threads
  • Author
    Posts
    • #14036
      NathanStoltz
      Participant

        Hello – I’m attempting to take the double_logger.ino file and modify it for my needs – and would appreciate a little help. I’ve been logging pretty well to MMW with the basic logging_to_MMW.ino using a XB3-C-A2-Ut-001 with Hologram Sim card, primarily with a tipping bucket. What I’m hoping to do with the double logger is to log every 5 minutes onto SD card, but only send out every hour to not burn through data as quickly. The code I’ve been working in is below.

        A few things:

        As the code is right now, the “low” interval reading returns all-9999 instead of actual values. The “high” interval reading returns correctly. Any idea why the Low is not reading values correctly?

        I’ve tried, without success, to add a command under the High interval to send out and publish – using either logDataAndPublish command or publishDataToRemotes that’s used in the data_saving.ino example – without success. How can I best log and publish the High interval data?

        Finally, I believe that I’ll need to add a calculated variable for the tipping bucket data to sum the tips in the low interval readings between a high interval reading. I plan to set low=5 min and high = 60 min, so the high needs to report the rainfall in the last hour, not just the last 5 minutes. Does that make sense?

        Any help would be really appreciated. I’m no coder, so modifying a program get’s tricky for me quickly when it goes beyond aggregating what I need via copy-paste. Thanks!

         

         

         

      • #14037
        Jim Moore
        Participant

          @stoltzfus-12osu-edu/ I am in a similar situation with my low cost EC sensor station project.  I need to tweek the AtlasScientific sensor library to fit my hardware.  I am using an AtlasSci EZO circuit set for UART mode and a sensorex conductivity probe with internal RTD which I use to report temperature as well as correct the EC reading to 25ºC.

          I am a retired EE (Cornell BEE ’62) and not a comp Sci major and my preferred programing language is solder!  It would be great to be able to provide functional specs for a particular instance of sensor hardware and someone with a CompSci background  (Sara?) write the code for the Modular Sensors library that could be pasted into the DRWI_citSci.ino code.  I plan to post details in this forum under a new topic.

        • #14041
          Sara Damiano
          Moderator

            @w3asa – I don’t have formal training in computer science or even in electrical engineering.   I have one bachelor’s level degree – in chemistry.


            @stoltzfus-12osu-edu
            – two things with your code – and I completely understand why you did it the way you did – because the example did.  Blame the chemist.

            1 – you’re creating each variable 3 times instead of creating it once and then referring it to that same one later.  Each time you use that “new” keyword it creates a new variable, but you can only assign one variable of each type to a sensor so the other two weren’t getting updated.  In the double logging example it worked fine to create the variables with “new” right in the variable array because there was no overlap between the two arrays.  But that definitely wansn’t made clear and is confusing.  Anyway, to fix that change your variable arrays so that instead of creating a new variable they use the same variable that you already created up in the sensor sections like so:

            Since you only use the free RAM in one of the two arrays and battery in the other and you didn’t create either at the top, it’s fine to create them “new” in the array. If you decide you do want them in both or the look of the code bothers you, you can create the free ram and battery variables with statements like Variable *yourVarName = new ProcessorStats_FreeRam(&mcuBoard, ...) up in the processor section or elsewhere above.

            2 – Modems in ModularSensors are special; they don’t get updated like regular variables tied to a sensor.  The signal quality isn’t update until *after* you send out some data.  With each cycle, the signal strength saved to the csv and posted is actually the data from one cycle prior.  And if you never post data (you don’t) and explicitly update the modem metadata, the values don’t update.  You’d want to add publishing code like this in the “high” section right after turning off the SD card.

             

             

          • #14042
            Sara Damiano
            Moderator

              Also, just so you understand, ModularSensors isn’t powerful enough to “stash” recent data and save it for later.  So when you’re in production and only connecting once an hour, the data being sent on the hour will only be for that one point.  If for some reason there’s something wrong with the cellular connection in that moment it is supposed to send (thunderstorm, planetary alignment, the XBee being moody, etc), the data will not be sent.  The logger will not try to send out the back-log.  You’ll have to go fill in the gaps later from the SD card data.  It’s definitely on my list to get that working so we could send more data per connection and reduce the connection overhead to data ratio, but I haven’t figured it out yet.

            • #14049
              NathanStoltz
              Participant

                Sara and Jim, thanks for the replies! Sara, knowing where you are now without formal coding training give me to hope to continue persevering!

                Thanks for the code suggestions! The first – not creating new variables each time – worked great.

                The second, for uploading to MMW at first did not work, saying modemWake and updateModemMetadata was not defined. Since I hadn’t updated the libraries in long time, I updated all libraries. Doing that fixed the errors. However, when running the program I have been receiving only 504 responses. I checked the standard logging_to_MMW.ino program with the new library  of modular sensors (0.24.5), and receive some 504s, but mostly it seems successful 201s (earlier this week before updating libraries it worked every time over ~12hrs). Sara, any suggestion why I can’t successfully log in this double_logger program?

                Thanks!

              • #14052
                Jim Moore
                Participant

                  I noticed a similar issue regarding response codes when I first got my project uploading to MonMW.

                  Copy of my response on another forum thread:

                  Couple of questions:
                  I noticed several different response codes from the server – 20x(good) 40x(eror). Yet the data uploaded successfully in both cases. Are these related to signal strength or slow response of the server?

                  Data seems to be uploaded with a 2G signal of -95 dBm so what is the noise floor of the GPRSbee modems?

                  In attempting to solve the missing data issue I deleted GMI_EC2 and reconfigured it and when I uploaded data with the DRWI_CitSci code it showed all four variables. I tried several months ago and that didn’t work. Can anyone explain what’s going on? Perhaps the DRWI_CitSci code has some magic incantation that I missed in my earlier attempts!
                  Thanks

                • #14055
                  Sara Damiano
                  Moderator

                    @stoltzfus-12osu-edu – that is very interesting to know.  Do you happen to know exactly what version of the library you’d had prior to updating?  If not, how long (at least roughly) would the long time since you updated be?  A month or two? A year?  The specification for the post format hasn’t changed at all since it was first developed, but if there could be something in my code that changed about the way I create the post request that I’ve somehow overlooked and decreased the success rate, I want to track it down.

                  • #14056
                    NathanStoltz
                    Participant

                      <p class=”user-nicename”>@srgdamiano – I wish I had looked prior to updating. It was at least July/August of 19, if not late February/early March of 19. I played around with using a back-dated version of the modular sensors library (by specifying the version in the .ini file) to see if I could figure it out.</p>

                    • #14057
                      NathanStoltz
                      Participant

                        Sara – Do you have any insight/help on why I (mostly) have success why using the logging_to_MMW program, which uses the logDataAndPublish command, but not with the double_logger program using the code you provided? It seems to print out the same information  in serial monitor of what it’s trying to do in both cases, but only returns on 201 on the logging_to_MMW program (otherwise 504).

                        Thanks!

                      • #14058
                        Sara Damiano
                        Moderator

                          I waded back through my “git blame” and I don’t see any changes I’ve made to the request more recent than 8 months ago, where I removed one single space – from a double space in one of the headers.  The next most recent change was 15 months ago.  Is it possible your libraries were that old?

                        • #14059
                          NathanStoltz
                          Participant

                            No, I don’t think it could have been that long ago. I apologize if I led  you down an un-necessary rabbit hole – it very well could have been that the update to the library had no bearing on how the logging_to_MMW worked at all. It seemed to correlate, but I definitely have nothing definitive to point to.

                          • #14078
                            Sara Damiano
                            Moderator

                              No problem!  The wonder of git is that I actually could track back pretty quickly how long it had been since any changes were made and what those were.

                          Viewing 11 reply threads
                          • You must be logged in to reply to this topic.