Welcome to EnviroDIY, a community for do-it-yourself environmental science and monitoring. EnviroDIY is part of WikiWatershed, an initiative of Stroud Water Research Center designed to help people advance knowledge and stewardship of fresh water.
New to EnviroDIY? Start here

Newbie Mayfly Setup for RS485 sensor to internet

Home › Forums › Mayfly Data Logger › Newbie Mayfly Setup for RS485 sensor to internet

Tagged: , ,

Viewing 28 reply threads
  • Author
    Posts
    • 2018-09-27 at 1:16 PM #12441
      neilh20
      Participant

        Hi, I’m a newbie to Mayfly and looking for any hints on setting up an environment.

        I’m planning the software on how to do a low power RS485 with logging/SD and sending to the internet (wifi then …) … solar powered of course … which means low power needs to be builtin from the start.

        I’ve got my RS485wing board and proven the Pololu +12V boost. Thanks for the pointers from @aufdenkampe @fisherba @bschulz1701 (It behaved very well with a straight +5Vinput with my own capacitors/assembly process – no problem with an output of 143mA@12V for an input 400mA@5V).
        I’m interfacing to a Keller levelgage and nanolevel gages – I’m very familiar with RS485 /Keller having done it on a beaglebone/linux as well as in a Nuttx/Particile.WiFi modem setting. I have a Salaea logic anaylzer for the RS485 protocol debug – which is really nice.

        Just wondering if anybody has a Mafyly/Arduino framework to recommend? as well as any other IDE tools.
        I have installed Arduino App 1.8.15/Windows 10. I last used in 2015 so its evolved since then – but I remember going beyond the basics to setting up a project was painful. I really like Eclipse C++ and typically integrate with github/bitbucket straightaway.
        I hear PlatformIO/VScode is good.

        I really appreciate browsing the envirodiy examples, what a lot of team work. I’ve also been looking at frameworks on github.
        NorthernWidget seems to describe a lot https://github.com/NorthernWidget/ALog, but doesn’t mention Mayfly, though it defines a __AVR_ATmega1284p__
        So maybe I should try the examples first
        https://github.com/EnviroDIY/ModularSensors/tree/master/examples/logging_to_EnviroDIY
        It seems the Mayfly RS485wing is using ports to do a soft 9600Baud for RS485, so seems I should prove out that interface straight away.
        So possibly integrate in AltSoftSerial with
        https://github.com/EnviroDIY/SensorModbusMaster/blob/master/src/SensorModbusMaster.cpp
        and watch it on the Salae analyzer.

        Any thoughts and suggestions appreciated.

      • 2018-09-27 at 1:39 PM #12448
        fisherba
        Participant

          Hi Neil,

          I spent the day outside yesterday, adding more sensors to two existing stations, which included some RS485 sensors. I’ll share how I did it and that will hopefully help you along!
          The stations are: https://data.envirodiy.org/sites/WSP01/ and
          https://data.envirodiy.org/sites/WSP02/.

          My sensor station added RS485 sensors and a new implementation of hardware I2C for PaleoTerra redox probes (previous probes said they were I2C, but they all had the same address as the Real Time Clock on the Mayfly … and every other RTC, so we had to implement as software I2C, which ate up so many pins). I now have 11 sensors chiming happily away on one Mayfly at each of those stations. We’ll see how the battery holds up, they get a lot of shade.

          How I did this is the subject of a blog post in process. I use PlatformIO as my IDE. What *broke* me and forced me into using PlatformIO was managing libraries for the Modular Sensor sketch. And I don’t regret learning to use PlatformIO at all (but other IDEs exist, I just don’t know them by name… pick one!). The Modular Sensor library is the only way I know how to do all the things we need for environmental monitoring (I’m an end-user, not a coder or an engineer). The Modular library is designed to
          * get multiple sensors to run with energy efficiency,
          * with solar charging and telemetry (Northern Widget hasn’t supported telemetry yet, but they are working on it),
          * and more recently with calculated variables (like water level corrected for atmospheric pressure).

          This sketch is the starting point that does it all:
          https://github.com/EnviroDIY/ModularSensors/tree/master/examples/logging_to_EnviroDIY

          Note: There’s a Keller Modbus sensor in this sketch, that might require slight modifications to get it running for your Keller sensors. Keller was very supportive of @aufdenkampe developing Arduino-framework compatibility for their sensors.

          I see that with Keller sensors you won’t need atmospheric pressure corrected values to log, but for those who would this sketch has that capacity and it’s the base of the example that I’m sharing below:
          https://github.com/EnviroDIY/ModularSensors/tree/master/examples/baro_rho_correction

          This is even more complicated than the logging sketch because in order to calculate variables from two separate sensors, @srgdamiano had to break out some of the functions.

          After you peek at those, then look at one of the sketches that I deployed yesterday. It’s modified from baro_rho_correction and it has some extra stuff in the loop because there’s an energy bleed from the RS485 wing. The bleed happens because when the modbus pins are left high, which happens because modbus stop bits are high, and the power is off to the module (the Wagoneer RS485 widget), it bleeds.

          Here’s the sketch.

          Here’s the PlatformIO ini file that I used to compile/build the sketch, which is how you manage libraries in this IDE.

          That’s a lot to take in… I expect you will have next questions! Meanwhile, this is a good push for me work on the blog post that I hope will help people get started in PlatformIO (or their IDE of choice) and Modular Sensors.

          Beth

        • 2018-09-27 at 8:02 PM #12463
          neilh20
          Participant

            Thanks @fisherba..
            Great to see your code..

            I’m trying though to see how I get up and going with PlatformIO

            https://github.com/EnviroDIY/ModularSensors/tree/master/examples/logging_to_EnviroDIY
            There is a platformio.ini

            I’ve forked and then cloned into my local machine such that I have
            C:\Users\neilh77\git\envirodiynh\ModularSensors\examples\logging_to_EnviroDIY

            I’ve downloaded the Visual Studio, and enabled the Platformio plugin
            It has arduino
            C:\Users\neilh77\.platformio\packages\framework-arduinoavr\cores\arduino
            I cloned;
            C:\Users\neilh77\git\platform-atmelavr\examples\arduino-blink>
            modified the .ini board=mayfly and ran and downloaded OK

            Then …. prompted by a lot of trial and error …4hrs of random typing and with sporadic glazed reading of .htmls …. finally this seemed to be the effect

            envirodiynh\ModularSensors\examples\logging_to_EnviroDIY
            I created a directory logging_to_EnviroDIY
            I took the logging_to_EnviroDIY.ino and renamed it into logging_to_EnviroDIY\logging_to_EnviroDIY.cpp

            then at the TERMINAL
            C:\Users\neilh77\git\envirodiynh\ModularSensors\examples\logging_to_EnviroDIY>
            entered
            >platformio run

            and something automagical seem to happen, getting libraries
            and it built.

            Compiling .pioenvs\mayfly\lib31b\EnviroDIY_ModularSensors_ID1648\YosemitechParent.cpp.o
            Compiling .pioenvs\mayfly\lib3ac\AltSoftSerial\AltSoftSerial.cpp.o
            Compiling .pioenvs\mayfly\libff4\SoftwareSerial_ExtInts\SoftwareSerial_ExtInts.cpp.o
            Archiving .pioenvs\mayfly\libFrameworkArduinoVariant.a
            Compiling .pioenvs\mayfly\FrameworkArduino\CDC.cpp.o
            Archiving .pioenvs\mayfly\lib3ac\libAltSoftSerial.a
            Archiving .pioenvs\mayfly\lib31b\libEnviroDIY_ModularSensors_ID1648.a
            Archiving .pioenvs\mayfly\libff4\libSoftwareSerial_ExtInts.a
            Compiling .pioenvs\mayfly\FrameworkArduino\HardwareSerial.cpp.o
            Compiling .pioenvs\mayfly\FrameworkArduino\HardwareSerial0.cpp.o
            Compiling .pioenvs\mayfly\FrameworkArduino\HardwareSerial1.cpp.o
            Compiling .pioenvs\mayfly\FrameworkArduino\HardwareSerial2.cpp.o
            Compiling .pioenvs\mayfly\FrameworkArduino\HardwareSerial3.cpp.o
            Compiling .pioenvs\mayfly\FrameworkArduino\IPAddress.cpp.o
            Compiling .pioenvs\mayfly\FrameworkArduino\PluggableUSB.cpp.o
            Compiling .pioenvs\mayfly\FrameworkArduino\Print.cpp.o
            Compiling .pioenvs\mayfly\FrameworkArduino\Stream.cpp.o
            Compiling .pioenvs\mayfly\FrameworkArduino\Tone.cpp.o
            Compiling .pioenvs\mayfly\FrameworkArduino\USBCore.cpp.o
            Compiling .pioenvs\mayfly\FrameworkArduino\WInterrupts.c.o
            Compiling .pioenvs\mayfly\FrameworkArduino\WMath.cpp.o
            Compiling .pioenvs\mayfly\FrameworkArduino\WString.cpp.o
            Compiling .pioenvs\mayfly\FrameworkArduino\abi.cpp.o
            Compiling .pioenvs\mayfly\FrameworkArduino\hooks.c.o
            Compiling .pioenvs\mayfly\FrameworkArduino\main.cpp.o
            Compiling .pioenvs\mayfly\FrameworkArduino\new.cpp.o
            Compiling .pioenvs\mayfly\FrameworkArduino\wiring.c.o
            Compiling .pioenvs\mayfly\FrameworkArduino\wiring_analog.c.o
            Compiling .pioenvs\mayfly\FrameworkArduino\wiring_digital.c.o
            Compiling .pioenvs\mayfly\FrameworkArduino\wiring_pulse.S.o
            Compiling .pioenvs\mayfly\FrameworkArduino\wiring_pulse.c.o
            Compiling .pioenvs\mayfly\FrameworkArduino\wiring_shift.c.o
            Archiving .pioenvs\mayfly\libFrameworkArduino.a
            Linking .pioenvs\mayfly\firmware.elf
            Checking size .pioenvs\mayfly\firmware.elf
            Building .pioenvs\mayfly\firmware.hex
            Memory Usage -> http://bit.ly/pio-memory-usage
            DATA: [====== ] 64.3% (used 10537 bytes from 16384 bytes)
            PROGRAM: [======== ] 80.4% (used 104532 bytes from 130048 bytes)

            WOW WOW

              WOW

            – YaaaaaaYippppeeee

            guess I try looking at the code now and see what its supposed to do …. but that might be Saturday

          • 2018-09-28 at 1:10 PM #12464
            fisherba
            Participant

              Hi Neil,

              You renamed the .ino file to a .cpp?

              Sorry I didn’t have time to explain what all of the things meant. 4 hours of glazed reading sounds painfully familiar.

              My platformio.ini file is in the root directory for my PlatformIO “project”. It has to be moved from the sketch folder to the root directory. Sara puts an example ini file in the sketch folder so we don’t have to guess which dependencies are needed for the sketch (which I really appreciate, but it seems to be a source of confusion for new users).

              Here’s where my sketch is located:
              /~/Documents/Arduino/EnviroDIY_deployments/deploy-BAF/logging_to_EnviroDIY/logging_to_EnviroDIY.ino

              And for that sketch, my ini file is located here:
              /~/Documents/Arduino/EnviroDIY_deployments/platformio.ini

              To “build” (aka “compile” and I sill think of it as compile, as does your terminal monitor!) the sketch, PlatformIO will use the instructions I give it in the ini file. When the build occurs, PlatformIO will fetch and write your libraries and write them to a hidden directory (at least as Atom renders PlatformIO). To see the hidden files, I go into Settings>Packages>tree-view and I uncheck the “Hide Ignored Names” default. This will reveal several folders in your root directory that were built based on your instructions in the ini file, including .git, .pioenvs, and .piolibdeps.

              My ini file looks something like this (and I’ll break it down below):

              ; PlatformIO Project Configuration File
              ;
              ; Build options: build flags, source filter
              ; Upload options: custom upload port, speed and extra flags
              ; Library options: dependencies, extra library storages
              ; Advanced options: extra scripting
              ;
              ; Please visit documentation for the other options and examples
              ; https://docs.platformio.org/page/projectconf.html

              [platformio]
              ;src_dir = deploy-Limno/MeteoTsu-1
              ;src_dir = deploy-BAF/UST/Bixby4c
              ;src_dir = deploy-BAF/testsketches/Yosemitech_utilities/ChangeSlaveID
              ;src_dir = deploy-BAF/testsketches/Yosemitech_utilities/GetSlaveID
              ;src_dir = deploy-BAF/YosiConductivity
              src_dir = deploy-BAF/WSP01
              ;src_dir = deploy-BAF/testsketches/onewiresearch
              ;src_dir = deploy-BAF/baro_rho_correction

              [env:mayfly]
              platform = atmelavr
              framework = arduino
              board = mayfly
              lib_ldf_mode = deep
              lib_ignore = RTCZero
              lib_deps =
              ; Using ModularSensors *master* branch as of May 18, 2018: v0.11.7
              ; EnviroDIY_ModularSensors@=0.11.6
              https://github.com/PaulStoffregen/AltSoftSerial.git
              https://github.com/EnviroDIY/SoftwareSerial_ExternalInts.git
              ;Using ModularSensors *PaleoTerraRedox* branch as of Sept 18, after hardware I2C with variable base correction
              https://github.com/EnviroDIY/ModularSensors.git#5928eca124aa76d6a3bd818c295df652422975fc

              In this syntax, the semicolon is the comment out character and it *must* be at the beginning of the line with no spaces in front if it or you will think your computer is blowing up.

              Here’s how you tell your instructions to PlatformIO:

              [platformio]: Tells PlatformIO where to find your sketch (your .ino file). Don’t point to a file name, point to the directory. Remember how the Arduino IDE required you to have your sketch in an enclosing folder of the same name? Still true in PlatformIO. Notice that I like to keep several directories handy here, but I can build only one directory (the rest are commented out).

              [env:mayfly]: This is your virtual environment. PlatformIO knows the Mayfly board (and something like 500 other boards). There are several things to specify in the environment:

              platform = this is the type of microcontroller you are using. The Mayfly (and probably most Arduinos?) are AVR, which is a family of microcontrollers developed by Atmel.

              framework = arduino. This one makes sense, but apparently you can use multiple frameworks.

              board = mayfly

              lib_def_mode = This is how you want PlatformIO to search for library dependencies. The mode options are listed here. I’ve been directed to use “deep” and that seems to include everything we need.

              lib_ignore = Exclude this library.

              lib_deps = This is where we point to the URLs of GitHub repositories. We can tell PlatformIO to use the most recent version of the library (syntax matters), such as anything greater than or equal to version 0.12.2:
              EnviroDIY_ModularSensors@>=0.12.2

              Or you can point to a specific commit on GitHub, which allows you to point to development branches. My recent deployment used some new work that hadn’t been merged into the Master branch, so I directed it to the development branch. When I do this, I make a note about the date and any memorable aspects of that commit, so I don’t have to go back and read the commits on GitHub to remember why I selected that. I also keep track of which commits I used when I deploy a station (in a readme.md file in my deployment repository) so I can replicate it if needed:
              ;Using ModularSensors *PaleoTerraRedox* branch as of Sept 18, after hardware I2C with variable base correction
              https://github.com/EnviroDIY/ModularSensors.git#5928eca124aa76d6a3bd818c295df652422975fc

              Not sure why they must be listed separately, but currently the Modular Sensors library requires these libraries:
              https://github.com/PaulStoffregen/AltSoftSerial.git
              https://github.com/EnviroDIY/SoftwareSerial_ExternalInts.git

            • 2018-09-30 at 10:29 PM #12489
              Sara Damiano
              Moderator

                I’m sorry for not chiming in earlier.

                PlatformIO is definitely a much, much better system for programming than the Arduino IDE. It’s much more powerful and the library manager is very good. I’m running it on Atom (instead of VSCode) but it’s the same beast underneath.

                The ModularSensors library is designed to be able to communicate with a whole bunch of sensors and to get the data from them onto an SD card and up to the network. I’ve spent a lot of time optimizing the timing of the variables in the arrays to make it very flexible while minimizing power and data consumption. I decided when I started building the library to create it so that the library itself would not need to be modified by users (ie, no library config file) but that unfortunately does mean that you must have the sub-libraries for every supported sensor installed on your computer in order to compile the library.

                The example programs are gigantic and scary because they contain the code for every single sensor that’s supported and a lot of extra notes about each one. You’d never want to really run the examples as they stand.

                Ok, so to get going in VSCode/PlatformIO:

                – Open VSCode, wait for it to load up. Make sure PlatformIO is installed.
                – Click on the little house on the left in the bottom blue bar (open PlatformIO home)
                – In the PlatformIO home window, click “New Project”
                – A window should pop up – give your project a name, select “EnviroDIY Mayfly” as your board, “Arduino” as your framework, and put it wherever you want on your PC, preferably in a brand new folder all of its own.
                – Be patient while the new project is set up.
                – Now, look at the new folders that were created (using windows explorer or VSCode). Find the folder named “src”. It will have a file “main.cpp” in it. Delete “main.cpp” (HINT: By right clicking on a file in VSCode and picking “Reveal in Explorer” you can get right to the folder.)
                – Download the example you want from the modular sensors library by navigating to it and then clicking “raw” and saving the page as a ino file. Put that ino file in the “src” directory where main.cpp used to be.
                – Now, in the folder above the src file, a platformio.ini file should have been created. Open that platformio.ini in VSCode.
                – Go back to GitHub and find the platformio.ini from the example and open it up.
                – Copy the section under the heading [env:mayfly] from the platformio.ini file on GitHub into the one on your computer. Save the file.
                – Gears should start turning as soon as you save the platformio.ini. A .piolibdeps folder should appear and more libraries should appear inside of it. If this doesn’t happen within a minute or so, force it by clicking on the “clean” button that looks like a trash can on the bottom left blue bar. (The clean functionality both removes any pre-compiled library files (these are usually hidden) and downloads any new libraries and their dependencies as listed in the platformio.ini file.
                – Go to your ino program. Scroll through it and delete the chunks for sensors that don’t apply to you. Change the sensor addresses and pins to what you really have. Save.
                – Click on the check mark in the bottom blue bar. This does “platformio run” – ie, it compiles everything but does not upload. Look for any compiler errors. Hopefully you don’t have any. If you do.. first check for missing semi-colons that you accidentally deleted when customizing your pins. Dig deeper from there.
                – To upload to your board, attach it to the computer and click on the arrow button between the check mark and the cloud. If you only have one serial device connected, the upload will automatically try to go to that. If you have more than one connected, you can make sure it goes to the right one by adding the line “upload_port = COM##” in the [env:mayfly] section of your platformio.ini file.
                – Check that everything is running smoothly by watching from a serial monitor. You can access the one built in with VSCode/PlatformIO by clicking the button that looks like a power cord in the blue bar. I think all of the examples have the serial port running at 115200 baud.

                That’s kind-of whirl wind and not very detailed, but I hope that gets you going, if you haven’t gotten farther than that already on your own!

                ~Sara

              • 2018-09-30 at 10:39 PM #12490
                Sara Damiano
                Moderator

                  Oh. Oops, apparently I didn’t refresh and see Beth’s latest response. She has much better detail than me. If you use the default “src” directory, you don’t need the src_dir = in your platformio.ini file. But you can only have a single ino in that folder, so if you want multiple programs for testing, you’ll need to specify the directories.

                  The two serial libraries are listed separately in the platformio.ini file because ModularSensors actually *doesn’t* require them, but most of the examples do. If you’re working with RS485 sensors and you’re also using some sort of remote communication, you’re almost certainly going to want to be using AltSoftSerial for the Serial communication with the sensors. AltSoftSerial is the best non-hardware serial port I’ve found. The SoftwareSerial_externalInts is pretty awful at best. NeoSWSerial is another workable alternative. There’s more detail in the ReadMe for ModularSensors about the serial port problems: https://github.com/EnviroDIY/ModularSensors#notes-on-arduino-streams-and-software-serial

                • 2018-10-01 at 11:08 PM #12521
                  neilh20
                  Participant

                    Gosh thanks for the info. Definitely very useful.

                    Sara some amazing classes you’ve put together. Thankyou Thankyou.

                    They really hummed for me. Still amazed the way they hang together – that’s the fun of optimizing it makes it look simple.
                    I usually add a serial debug with a time stamp to watch how long it takes.
                    Just wondering – I can see some debug capability built in MS_DBG macro – is it possible to enable it, or the serial debug in some way.
                    I tried adding to platformio.ini
                    build_flags = -DDEBUGGING_SERIAL_OUTPUT
                    and it compiled some modules, and then came up with an error somewhere – making it look like it should only be done on specific files.?

                    For the keller nanolevel it wouldn’t respond to the same modbus requests as Keller acculevel 🙁 – so I ended up having to change the message structure to what I had used on another project.

                    I expanded “enum kellerModel” and introduced _model to the kellerMobdus – following yosemitech. Its some very ace class structures…

                    So effectively I’ve created a KellerNanolevel.h and modified KellerParent.h/cpp and KellerModbus.cpp/h. Would you like me to submit a pull request for it.?
                    PlatformIO is very flexible, and to get started (hadn’t read your emails) I ended up with
                    Documents\Arduino\env01\nanolevel\logging_to_EnviroDIY.ino and platformio.ini
                    In platformio.ini
                    [platformio]
                    ;src_dir = logging_to_EnviroDIY
                    src_dir = .

                    Then created
                    Documents\Arduino\env01\nanolevel\lib\KellerNanolevel\src – where it automatically looked for source files
                    and then copied in files and gave them unique names to make changes
                    KellerNanolevel.h KellerNanoParent.h KellerNanoParent.cpp

                    Cheers

                    So an output with my new sensor enabled and the internal sensors.

                    Data Logger: nh05a ExternalVoltage MaximDS3231 KellerNanolevel KellerNanolevel EnviroDIY Mayfly
                    Data Logger: nh05a voltage temperatureDatalogger gaugeHeight temperature Free SRAM batteryVoltage
                    Data Logger: nh05a volt degreeCelsius meter degreeCelsius Bit volt

                    Date and Time in UTC-8 extVoltage BoardTemp kellerHeight kellerTemp FreeRam Battery
                    1/3/2000 15:22 5.89625 23.5 -0.0053 22.35 12758 13.387
                    1/3/2000 15:30 5.89125 23.5 -0.0053 22.37 12758 13.387
                    1/3/2000 15:32 5.89625 23.5 -0.0052 22.39 12758 13.387

                  • 2018-10-02 at 2:06 PM #12522
                    Sara Damiano
                    Moderator

                      Please, do send in a pull request! I really deeply appreciate other people taking the time to look through my code and giving any suggestions they might have.

                      You should be able to use the build flag “build_flags = -D DEBUGGING_SERIAL_OUTPUT=Serial” but that might give you way, way more than you want to see. There is a LOT of debugging output possible in the library. I’d suggest you open the header (*.h) file for whatever you’re trying to debug and un-comment the define in that header only. The Keller Parent also has a “deep debugging” output which will spit out the raw modbus commands.

                    • 2018-10-02 at 6:35 PM #12528
                      neilh20
                      Participant

                        Hi @srgdamiano – wonderful to work through the code. Very readable and nicely laid out. Very nice to step through the classes. I’m mostly used C, TinyOS/nesC, Python and now JS. But great to see realtime cpp -all on mega1284 device!!!

                        DEBUGGING: that was what I was looking for. after seeing a line in the macro file I was guessing some trick out there… now I see in KellerParent.h
                        // #define DEBUGGING_SERIAL_OUTPUT Serial
                        // #define DEEP_DEBUGGING_SERIAL_OUTPUT Serial
                        #include “ModSensorDebugger.h”

                        So a general question – it looks like when it comes time to start sensor reading,
                        there is no opportunity to say connect to the cloud after every Xth sample or after 4hrs ?
                        Typically I’ve seen with 3G modems that it spends 1minute at high current (0.7A) finding the tower and setting up the tcp link. Then the data is pushed fast and closes down.
                        So just wondering about that.

                        Also wondering if a wireless connection fails, is there a fall back method for pushing the readings later.
                        I have had some sites in the past that did not connect for two weeks as they are on the edge of the zone, and it gets a cloudy for two weeks and no RF.

                      • 2018-10-02 at 7:14 PM #12529
                        Sara Damiano
                        Moderator

                          @neilh – Thank you!

                          You can connect and send data at a different rate than you log it by essentially creating two different logger objects and setting them to different intervals. The “data_saving” and “double_logger” examples each do slight variants on this right now. You just can’t use the easy “log” function like in the data_to_envirodiy example.

                          Right now, there isn’t any fall back for sending missed data. Data gets sent right after it’s created or never. It is all logged to a SD card though! I’ve thought about trying to create a buffer of un-sent data, but I haven’t come up with a decent way of doing it and it hasn’t been a high priority for me. If you have ideas, though, let me know!

                        • 2018-10-03 at 12:56 AM #12532
                          neilh20
                          Participant

                            Interesting – I would treat the logging the SD as just a step on the way to having it registered on the remote site database/cloud – whatever the transmission method.
                            I’ve done it in the past on a serial SPI flash, with two pointers, one for the next log sample and then a following pointer for what was last
                            ACKed remotely. It works because its direct access on the serial flash – not a file. This also allowed a UDP type packet in a mesh XPB 900Mhz network

                            Since ram is likely to be a limitation, and persistent storage is SD card, I would think it needs be stored on the SD card.
                            For an SD, maybe a way would be if the sensor reading transmission isn’t successful (ie never attempted, or not ACKed from the remote site) for whatever reason, its stored in a separate SD file <reading_staged.json> .
                            The logging happens as per defined schedule. The wireless connection also has an algorithm for connection attempts. (ie attempt connect every 4th logging interval, or every 24hrs, AND only if there is adequate power [battery voltage] -so makes extended solar loss more manageable)
                            Then when a connection attempt to the remote site is successful, if the file <reading_staged.json> exists, the current reading is written on to the end of the file. Then renamed to <reading_upload.json> and transmitted sequentially from the file.
                            If the connection is lost halfway through, then any undelivered records are written to <reading_staged.json>, and delete <reading_upload.json>.
                            Just an idea.

                            What I’ve also found useful is a <record_num>, and that has enabled me visually audit on the cloud database to see if there are missing records.

                            Well easy to have ideas. Still working my way through.
                            I get the Digi WiFi XB2B-WFWT-001 and the new Digi CAT-M1 XB3-C-A2-UT-001 Xbees on Thursday. THe Cat-M1 is very exciting from the power point of view as its designed for data delivery and faster to connect to the cell power from power off.

                            Cheers

                          • 2018-10-04 at 4:16 PM #12579
                            neilh20
                            Participant

                              Hi
                              I’m running into a low power issue with the sensors/RS485 turned off.
                              There is a backfeed going on, which leaves SW_3.3V at about 2V and LED3 comes on.

                              I think @aufdenkampe gave me a hint of this and described a solution
                              https://github.com/EnviroDIY/ModularSensors/issues/140 in discussion with @srgdamiano

                              However I can’t find it in the release I have – and I’m not sure I understand if @aufdenkampe solution in 140 is releasable since its in LoggerEnviroDIY.cpp
                              That is the RS485 serial port enable bits have to follow the SW_3V3 as is turned on/off.
                              Seems to me this should happen in the sensor power up and down for the ModbusParent? – though I haven’t been able to find where power up/down is implemented yet. Perhaps I haven’t dug deep enough
                              So modbusParent power up would be
                              // Restart the stream for the modbus sensors, because we set pins LOW, below
                              modbusSerial.begin(9600);

                              and modbusParent power down
                              // Reset AltSoftSerial pins to LOW, to reduce power bleed on sleep,
                              // because Modbus Stop bit leaves these pins HIGH
                              digitalWrite(5, LOW); // Reset AltSoftSerial Tx pin to LOW
                              digitalWrite(6, LOW); // Reset AltSoftSerial Rx pin to LOW

                              The issues are closed – so would they be migrated into the main ?
                              https://github.com/EnviroDIY/ModularSensors/issues/140
                              https://github.com/EnviroDIY/ModularSensors/issues/148 same issue @fisherba
                              many thanks for insights.

                            • 2018-10-04 at 5:03 PM #12580
                              Sara Damiano
                              Moderator

                                The modbus power up and down *can’t* include the begin or the pin settings because the modbus functionality is built around any Arduino stream type object. You can use a modbus sensor connected to anything that you can communicate with as an Arduino stream – that means HardwareSerial, SoftwareSerial, Client, etc. If you’re not using Serial1 for a modem, I’d actually strongly recommend you use it and not AltSoftSerial. Stream is a pure virtual class and it doesn’t actually have a begin function. The begin() functions are implemented by specific streams. And unless you are using the AltSoftSerial stream instance on a Mayfly or Mbili board, that digital write low won’t work.

                                The powerUp and powerDown functions themselves are implemented in SensorBase. Though, actually, in the newest branch I’ve moved the modbus ones (Yosemitech/Keller) into their direct parents to allow for separate power to the sensor and an RS485 adapter. I’m about to submit a pull request with that and some other updates.

                              • 2018-10-04 at 5:05 PM #12581
                                Sara Damiano
                                Moderator

                                  Oh, and a warning on the Digi LTE-M XBee: I can’t get mine to work on the Hologram network that we’ve been using with 2G. Other people say they have, but I can’t seem to get mine to work.

                                • 2018-10-04 at 6:14 PM #12582
                                  neilh20
                                  Participant

                                    Hi @srgdamiano –

                                    SensorBase Class – I’m a C guy trying for some class – but I’ll take the power switching where ever those in the higher classes deem it 🙂 Let me know where I could pull from when its available.

                                    I hear you about AltSoftSerial – I got a gray hair just thinking about it – but its the way RS485 wingboard plugs into the Mayfly and so far its worked. It seems awfully dependent on playing nice with interrupts.

                                    Now there is such good software available – if the AltSoftSerial fails I’ll have to think of a hardware solution – possibly moving the RS485 from connector J4 to J3….. but even then start to get into limitations with serial ports – one is needed for debug and one is needed for XBEE .
                                    Oh well the world of Arm with 4+ serial ports is in another universe – I have my favourite MKL27Z256 – builtin USB, RTC, ADC16bit, 32Ksram – and costs same mega1284. Same family as TeensyLC, but has ROM bootloader like mega1284. I have one running on another board – Oh well.!!

                                    Thanks for the headsup on LTE-M – good to hear that some people have it working with Hologram network – I understand from another project that I’m on that LTE-M’s coverage is extending. I haven’t found a map yet of coverage – but will keep ears open.
                                    I imagine you know about the 2G network changes https://www.envirodiy.org/wild-wireless-world-2g-or-3g/

                                  • 2018-10-05 at 8:22 PM #12587
                                    neilh20
                                    Participant

                                      Just wondering is anybody is using the Digi Xbee S6 module.
                                      I’m trying it as my first test module
                                      I have an open guest wifi that I’ve used.
                                      It initialled configured it -.
                                      const char *wifiId = “MyGuest”; // The WiFi access point, unnecessary for gprs
                                      const char *wifiPwd = “”; // The password for connecting to WiFi, unnecessary for gprs
                                      It powered up, and didn’t appear to work.
                                      So enabled logging that I could find.
                                      it became all strange..
                                      I put it on XCTU/XBIB to read it and could see it did the WiFiID – but then it started behaving strangely
                                      So I’ve changed to this
                                      #if defined(TINY_GSM_MODEM_XBEE)
                                      const long ModemBaud = 9600; // Default for XBee is 9600, I’ve sped mine up to 57600
                                      const int8_t modemSleepRqPin = -1;//23; // Modem SleepRq Pin (for sleep requests) (-1 if unconnected)
                                      const int8_t modemStatusPin = -1;//BEECTS;//19; // Modem Status Pin (indicates power status) (-1 if unconnected)
                                      const int8_t modemVCCPin = -1; // Modem power pin, if it can be turned on or off (-1 if unconnected)
                                      ModemSleepType ModemSleepMode = modem_sleep_reverse; // How the modem is put to sleep

                                      and from reset I get the following
                                      with the last two sets of commands goign to the XBEE WiFi
                                      ATAC\r response OK\r
                                      ATHS\r 601\r
                                      ATCN\r OK\r
                                      then
                                      +++ response OK\r
                                      ATS1\r OK\r
                                      ATS0200\r OK\r
                                      ATWR\r OK\r
                                      ATAC\r OK\r
                                      ATCN\r OK\r
                                      then nothing ….
                                      It looks like it looses or is reinitialized from knowing its an XBEE Wifi
                                      Any suggestions on how to configure Xbee WiFi. I haven’t worked through the AT cmd set as I think I’m doing something wrong 🙁

                                      —————-The terminal output from reset—
                                      Now running logging_to_EnviroDIY.ino on Logger nh06b
                                      Logger timezone is UTC-8
                                      RTC timezone is UTC
                                      Creating a new on/off method for the Digi XBee with power on pin -1 status on pin -1 and on/off by holding pin -1 low.
                                      Creating a new TinyGSM modem and client for the Digi XBee…
                                      Initializing Digi XBee…
                                      [1863] ### Modem: Digi XBee® Wi-Fi
                                      … Complete!
                                      Modem attached!
                                      Pin 8 set as LED alert pin
                                      Pin 21 set as testing mode entry pin
                                      Registration token set!
                                      Sampling feature UUID set!
                                      Current RTC time is: 2000-01-02T18:01:54-08:00
                                      Setting up logger nh06b to record at 10 minute intervals.
                                      This logger has a variable array with 7 variables, of which 7 come from 4 sensors and 0 are calculated.
                                      Successfully connected to SD Card with card/slave select on pin 12
                                      Opened existing file: nh06b_2000-01-02.csv
                                      Data will be saved as nh06b_2000-01-02.csv
                                      This logger is also tied to a NO MODEM SELECTED for internet connectivity.
                                      Turning modem on.
                                      Skipping modem in sensor power up!
                                      Skipping modem in sensor power down!
                                      Attempting to synchronize RTC with NIST
                                      This may take up to two minutes!

                                      Waiting up to 5 seconds for modem to respond to AT commands…

                                      Modem does not respond to AT commands!
                                      Turning modem off.
                                      Push button on pin 21 at any time to enter sensor testing mode.
                                      Logger setup finished!
                                      ——————————————

                                      Current Unix Timestamp: 946836120
                                      Logging interval in seconds: 600
                                      Mod of Logging Interval: 120
                                      Number of Readings so far: 0
                                      Mod of 120: 0
                                      Time marked at (unix): 946836120
                                      year: 2000
                                      month: 1
                                      date: 2
                                      hour: 18
                                      minute: 2
                                      second: 0
                                      Time marked at [char]: 2000-01-02T18:02:00-08:00
                                      Time to log!

                                    • 2018-10-08 at 10:52 AM #12589
                                      Sara Damiano
                                      Moderator

                                        Your setup put the Bee to go into deep sleep, but you didn’t assign a pin to be the wake-up pin. So the Bee isn’t waking up anymore. You need to either set the modem to be always on (ModemSleepMode = modem_always_on) or put a pin in for modemSleepRqPin (probably 23 if you’re using a Mayfly). If you want the modem to not sleep, you will have to use XCTU or manually send the commands to the Bee to turn off sleep mode.

                                        NOTE: The next version of this library coming out soon will change this set-up.

                                      • 2018-10-09 at 1:22 PM #12592
                                        neilh20
                                        Participant

                                          Hi @srgdamiano, Thanks for the headsup on new enviroDIY/TinyGSM release. I can see init changes on https://github.com/EnviroDIY/TinyGSM/branches

                                          I did start with Mayfly setup:
                                          const char *wifiId = “MyGuest”; // The WiFi access point, unnecessary for gprs
                                          const char *wifiPwd = “”; // The password for connecting to WiFi, unnecessary for gprs
                                          const long ModemBaud = 9600; // Default for XBee is 9600, I’ve sped mine up to 57600
                                          const int8_t modemSleepRqPin = 23;//23; // Modem SleepRq Pin (for sleep requests) (-1 if unconnected)
                                          const int8_t modemStatusPin = 19;//BEECTS;//19; // Modem Status Pin (indicates power status) (-1 if unconnected)
                                          const int8_t modemVCCPin = -1; // Modem power pin, if it can be turned on or off (-1 if unconnected)
                                          ModemSleepType ModemSleepMode = modem_sleep_reverse; // How the modem is put to sleep

                                          and thats when it initialized and then hung. XCTU showed MyGuest, Encryption Enable, PK Passphrase empty (which it always shows as empty, it seems its a write only field for pwds!).
                                          So connecting a logic analyzer on XbeeWifi pins the modemSleepRqPin wasn’t requesting wake state and sending data to a sleeping Xbee – so after about 3hrs of tracing input to the XbeeWifi module I figured I was getting it wrong.

                                          So checking the updates pending on EnviroDIY/TinyGSM I figure I should use your new release (thanks for all the updates to vshymanskyy/TinyGSM) and then try again.
                                          So to try again with the Mayfly, I think should
                                          1) Reset the XBEE WiFi module to defaults via XCTU
                                          2) Set “logging_to_EnviroDIY.ino” to sleep mode as above and then Monitor the modemSleepRqPin and modemStatusPin and Tx & Rx
                                          and see if it works.

                                          I can go onto a different project until the new EnviroDIY/TinyGSM lib is released.
                                          Alternatively, if it would be useful to you, I could attempt a pre – release build of enviroDIY/TinyGSM (master or original?).

                                          Here is my current log
                                          <start log with pins enabled, and WiFi configured********************************>
                                          Now running logging_to_EnviroDIY.ino on Logger nh06c
                                          Logger timezone is UTC-8
                                          RTC timezone is UTC
                                          Creating a new on/off method for the Digi XBee with power on pin -1 status on pin 19 and on/off by holding pin 23 low.
                                          Creating a new TinyGSM modem and client for the Digi XBee…
                                          Initializing Digi XBee…
                                          [1863] ### Modem: Digi XBee® Wi-Fi
                                          … Complete!
                                          Modem attached!
                                          Pin 8 set as LED alert pin
                                          Pin 21 set as testing mode entry pin
                                          Registration token set!
                                          Sampling feature UUID set!
                                          Current RTC time is: 2000-01-06T11:00:22-08:00
                                          Setting up logger nh06c to record at 10 minute intervals.
                                          This logger has a variable array with 7 variables, of which 7 come from 4 sensors and 0 are calculated.
                                          Successfully connected to SD Card with card/slave select on pin 12
                                          Opened existing file: nh06c_2000-01-06.csv
                                          Data will be saved as nh06c_2000-01-06.csv
                                          This logger is also tied to a NO MODEM SELECTED for internet connectivity.
                                          Turning modem on.
                                          Skipping modem in sensor power up!
                                          Skipping modem in sensor power down!
                                          Attempting to synchronize RTC with NIST
                                          This may take up to two minutes!

                                          Waiting up to 5 seconds for modem to respond to AT commands…

                                          Modem does not respond to AT commands!
                                          Turning modem off.
                                          Clock interrupt!
                                          Push button on pin 21 at any time to enter sensor testing mode.
                                          Logger setup finished!
                                          ——————————————

                                          Cloÿ,HZ·terrupt!
                                          Clock interrupt!
                                          Current Unix Timestamp: 947156460
                                          Logging interval in seconds: 600
                                          Mod of Logging Interval: 60
                                          Number of Readings so far: 0
                                          Mod of 120: 60
                                          Not time yet.

                                        • 2018-10-09 at 9:09 PM #12593
                                          neilh20
                                          Participant

                                            Got Xbee Sleep access fixed.
                                            I’m now communicating to the Xbee Wifi and getting NIST time.
                                            Looks like an error on post.

                                            Fixed the Xbee Sleep access;
                                            envirodiy\ModularSensors\src\ModemOnOff.cpp
                                            bool heldOnOff::on(void)
                                            //digitalWrite(_modemSleepRqPin, HIGH);
                                            digitalWrite(_modemSleepRqPin,_isHighWhenOn) ;

                                            bool heldOnOff::off(void)
                                            //digitalWrite(_modemSleepRqPin, LOW);
                                            digitalWrite(_modemSleepRqPin,!_isHighWhenOn);

                                            <start log of post>
                                            \/—- Post Request to EnviroDIY —-\/
                                            POST /api/data-stream/ HTTP/1.1
                                            Host: data.envirodiy.org
                                            TOKEN: 7b19191b-b3d6-416d-9e07-4fbe549f2493
                                            Content-Length: 334
                                            Content-Type: application/json

                                            {“sampling_feature”: “b918f150-4fff-424d-813a-a69a808a93b6”, “timestamp”: “2018-10-09T16:58:00-08:00”, “3907922a-56fe-46f3-a56e-9de6b77d3679”: 23.50, “f40a9dff-2c54-43ae-a016-1c5095c188eb”: 0.0323, “7bc30855-ae08-4865-ab89-a8e17df93bfc”: 19.24, “2c58e64d-6b66-4d9e-b893-bfdb10b65426”: 4.457, “0cf94fc8-a5d2-4fbe-82f2-2a81650575a8 “: 0}

                                            [26306] ### Unhandled: r: nginx/1.10.3 (Ubuntu)

                                            Date: Wed, 10 Oct 2018 00:58:14 GMT

                                            th: 78

                                            Connection: keep-alive

                                            Vary: Accept

                                            X-Frame-Options: SAMEORIGIN

                                            Allow: POST, OPTIONS

                                            {“detail”:”JSON parse error – Expecting object: line 1 column 334 (char 333)”}HTTP/1.1 400 Bad Request

                                            Server: n

                                          • 2018-10-10 at 3:06 PM #12596
                                            Sara Damiano
                                            Moderator

                                              It looks like you mis-typed the last GUID’s; you put a space on the end of it. The parser doesn’t expect a space inside of the GUID, so it fails. The error is at character 333, which is the guilty space.

                                            • 2018-10-10 at 3:45 PM #12597
                                              neilh20
                                              Participant

                                                Ok thanks – great eye – I caught a couple of spaces when I pasted it in, but the last GUID was later
                                                Now I am posting and getting 201 back.

                                                So a question -if I’m going to submit updates – should I be working of the master branch or the develop branch?
                                                currently I have library from May 24th update of
                                                https://github.com/EnviroDIY/ModularSensors/blob/master/examples/logging_to_EnviroDIY/platformio.ini
                                                lib_deps =
                                                EnviroDIY_ModularSensors@>=0.12.2

                                                ………………
                                                Clock interrupt!
                                                Current Unix Timestamp: 1539171120
                                                Logging interval in seconds: 300
                                                Mod of Logging Interval: 120
                                                Number of Readings so far: 0
                                                Mod of 120: 0
                                                Time marked at (unix): 1539171120
                                                year: 2018
                                                month: 10
                                                date: 10
                                                hour: 11
                                                minute: 32
                                                second: 0
                                                Time marked at [char]: 2018-10-10T11:32:00-08:00
                                                Time to log!
                                                Local:RS485on–
                                                Turning modem on.
                                                nhTurning modem on by setting pin 23 to 0.
                                                Modem now on.
                                                Powering sensors…
                                                Skipping modem in sensor power up!
                                                Waking sensors…
                                                Updating sensor values…

                                                Waiting up to 5 seconds for modem to respond to AT commands…
                                                [12578] < 1 >
                                                [12582] ### AT:
                                                [12584] < 1 >
                                                [12593] ### AT: CN
                                                [12595] < 1 >
                                                [12814] < 1 >
                                                [12822] ### AT: AI
                                                [12830] ### AT: CN
                                                [12832] < 1 >
                                                No prior internet connection, attempting to make a connection.
                                                Waiting up to 5 seconds for modem to respond to AT commands…
                                                [13060] < 1 >
                                                [13064] ### AT:
                                                [13066] < 1 >
                                                [13074] ### AT: CN
                                                [13076] < 1 >

                                                Waiting up to 5 seconds for WiFi network…
                                                [13297] < 1 >
                                                [13303] ### AT: AI
                                                [13314] ### AT: CN
                                                [13316] < 1 >
                                                Sending credentials…
                                                [13535] < 1 >
                                                [13543] ### AT: EE 2
                                                [13547] < 1 >
                                                ……
                                                [13604] ### AT: WR
                                                [13635] < 1 >
                                                [13643] ### AT: AC
                                                [13647] < 1 >
                                                [13654] ### AT: CN
                                                [13658] < 1 >
                                                [13877] < 1 >
                                                [13883] ### AT: AI
                                                [13891] ### AT: CN
                                                ……
                                                [18872] < 1 >
                                                [18878] ### AT: AI
                                                [18888] ### AT: CN
                                                [18890] < 1 >
                                                … Connection failed
                                                Connecting to NIST daytime server to check connection strength…
                                                Connecting to 129.6.15.30…[19116] < 1 >
                                                [19124] ### AT: IP 1
                                                [19128] < 1 >
                                                [19146] ### AT: DL 129.6.15.30
                                                [19150] < 1 >
                                                [19159] ### AT: DE 25
                                                [19163] < 1 >
                                                [19169] ### AT: WR
                                                [19202] < 1 >
                                                [19208] ### AT: AC
                                                [19212] < 1 >
                                                [19218] ### AT: CN
                                                [19222] < 1 >
                                                …Success!
                                                Getting signal quality:
                                                [19550] < 1 >
                                                [19556] ### AT: LM
                                                [19566] ### AT: CN
                                                [19568] < 1 >
                                                RSSI: 162
                                                Percent signal strength: 0
                                                Get Values:
                                                Pressure_mbar:3.18
                                                Temp_C: 17.18
                                                Height_m:0.03
                                                Get Values:
                                                Pressure_mbar:3.17
                                                Temp_C: 17.16
                                                Height_m:0.03
                                                Get Values:
                                                Pressure_mbar:3.17
                                                Temp_C: 17.16
                                                Height_m:0.03
                                                Putting sensors back to sleep…
                                                Cutting sensor power…
                                                Skipping modem in sensor power down!
                                                Local:RS485off–
                                                Connecting to the Internet…

                                                Waiting up to 5 seconds for modem to respond to AT commands…
                                                [24643] < 1 >
                                                [24647] ### AT:
                                                [24649] < 1 >
                                                [24657] ### AT: CN
                                                [24659] < 1 >

                                                Waiting up to 50 seconds for WiFi network…
                                                [24879] < 1 >
                                                [24887] ### AT: AI
                                                [24895] ### AT: CN
                                                [24897] < 1 >
                                                … Connected with saved WiFi settings!
                                                Connecting to data.envirodiy.org…[25120] < 1 >
                                                [25145] ### AT: LA data.envirodiy.org
                                                [25288] ### AT: IP 1
                                                [25290] < 1 >
                                                [25313] ### AT: DL 129.123.41.212
                                                [25317] < 1 >
                                                [25325] ### AT: DE 50
                                                [25329] < 1 >
                                                [25335] ### AT: WR
                                                [25368] < 1 >
                                                [25374] ### AT: AC
                                                [25378] < 1 >
                                                [25387] ### AT: CN
                                                [25389] < 1 >
                                                …Success!

                                                \/—- Post Request to EnviroDIY —-\/
                                                POST /api/data-stream/ HTTP/1.1
                                                Host: data.envirodiy.org
                                                TOKEN: 7b19191b-b3d6-416d-9e07-4fbe549f2493
                                                Content-Length: 334
                                                Content-Type: application/json

                                                {“sampling_feature”: “b918f150-4fff-424d-813a-a69a808a93b6”, “timestamp”: “2018-10-10T11:32:00-08:00”, “3907922a-56fe-46f3-a56e-9de6b77d3679”: 21.25, “f40a9dff-2c54-43ae-a016-1c5095c188eb”: 0.0324, “7bc30855-ae08-4865-ab89-a8e17df93bfc”: 17.16, “2c58e64d-6b66-4d9e-b893-bfdb10b65426”: 3.972, “0cf94fc8-a5d2-4fbe-82f2-2a81650575a8”: 0}

                                                [26597] < 1 >
                                                [26605] ### AT: TM0
                                                [26609] < 1 >
                                                [26615] ### AT: WR
                                                [26648] < 1 >
                                                [26654] ### AT: AC
                                                [26658] < 1 >
                                                [26664] ### AT: CN
                                                [26669] < 1 >
                                                [26888] < 1 >
                                                [26896] ### AT: TM64
                                                [26900] < 1 >
                                                [26906] ### AT: WR
                                                [26939] < 1 >
                                                [26945] ### AT: AC
                                                [26949] < 1 >
                                                [26955] ### AT: CN
                                                [26959] < 1 >
                                                Closed TCP/IP.
                                                — Response Code —
                                                201
                                                Running a daily clock sync…
                                                Disconnecting from the Internet…
                                                Turning modem off.
                                                Turning modem off by setting pin 23 to 1.
                                                Modem now off.
                                                Successfully connected to SD Card with card/slave select on pin 12
                                                Opened existing file: nh06d_2018-10-10.csv

                                                \/—- Line Saved to SD Card —-\/
                                                2018-10-10 11:32:00,21.25,0.0324,17.16,3.972,0

                                                ——————————————

                                              • 2018-10-10 at 6:54 PM #12598
                                                Sara Damiano
                                                Moderator

                                                  Up to you whether you want to work with the develop or master branch. They are currently **not** compatible with each-other though, so if you’re flipping back and forth, you’ll need to adjust your code each time to match up. I think all the changes are in the modem structure.

                                                  I don’t know if the May code is compatible with the current master or not. It’s definitely not with the current develop.

                                                  The code in the develop branch is **theoretically** better than what’s up on master, but it’s never been run by anyone except me and never on a sensor outside of my building. I don’t want to push it to master and release it until someone (or hopefully several someones) can verify its functionality a bit.

                                                • 2018-10-11 at 2:41 PM #12599
                                                  fisherba
                                                  Participant

                                                    My 2-cents on which branch to use is that the *model* structure is that end users work from the master branch. @neilh20 might be overqualified as an end user, so if future readers show up on this thread for help, they may be mislead by the notion that develop is the better branch.

                                                    But the Modular branches have been so very active in development lately (woot!) that I (very end user) have deployed nearly everything from development branches. What I think @srgdamiano means by the develop branch being *better* is that it has new features. I’m outside of Sara’s building and have been one of the someones actively requesting and testing new features. But buyer beware, that if the features haven’t been merged to master, they may not be fully tested.

                                                    So the bottom line regarding which commit to use: use the github commit that matches wherever you pulled the example file. If your Git is pointing to master, use the master branch commit to compile(build) your sketch. If your Git is pointing to develop, use that branch. When Sara does her development work, she is very good about making sure the examples match the current features, and this means that compiling from other branches may not work.

                                                  • 2018-10-11 at 3:52 PM #12600
                                                    neilh20
                                                    Participant

                                                      Thanks for the status 🙂 very much appreciate its a a work in progress – sometimes it just figuring it out. Its great to have it visible on github and to get pointers.
                                                      For software stability, testing is everything, and the options for boards and modems are a matrix.
                                                      For the record using the Xbee WiFi S6 hybrid, the current master branch as of this date needs updates to ModemOnOff.cpp to manage the PowerOnOff
                                                      With those updates I ran the Xbee Wifi last night posting to data.envirodiy.org
                                                      However alot the changes/work that has been performed since May and ModemOnOff.cpp functions are merged into another class – and I’m still exploring what has changed – so will continue that when the changes are released.

                                                      I do appreciate everyone’s pointers – and also when there is a lot of work being released – sometimes the best for any outsider (me) is just to support with testing when it becomes appropriate, and of course lots of learning about the changes.

                                                    • 2018-10-11 at 8:11 PM #12601
                                                      neilh20
                                                      Participant

                                                        Just wondering if I should be able to build from the develop or whether there are dependencies that make it too complex.
                                                        If I need to just wait until it gets to master – then thats OK as well, but if its useful;

                                                        I’m using GitHub Desktop (latest 1.4.2)
                                                        I switch to ModuleSensors which map to https://github.com/EnviroDIY/ModularSensors.git and maps to my c: git\envirodiy\ModularSensors
                                                        Fetch origin shows just updated
                                                        I set current branch to develop
                                                        In Git Power shell I have
                                                        C:\Users\neilh77\git\envirodiy\ModularSensors\examples\logging_to_EnviroDIY [develop]> ls
                                                        Directory: C:\Users\neilh77\git\envirodiy\ModularSensors\examples\logging_to_EnviroDIY

                                                        Mode LastWriteTime Length Name
                                                        —- ————- —— —-
                                                        -a—- 10/11/2018 12:10 PM 44318 logging_to_EnviroDIY.ino
                                                        -a—- 10/11/2018 4:33 PM 760 platformio.ini
                                                        -a—- 10/4/2018 10:25 AM 917 ReadMe.md

                                                        I start up VS with PlatformIO
                                                        File->Open Folder->C:\Users\neilh77\git\envirodiy\ModularSensors\examples\logging_to_EnviroDIY
                                                        I then press the icon “trash can” PlatformIO Clean
                                                        > Executing task: C:\Users\neilh77\.platformio\penv\Scripts\platformio.exe run –target clean <

                                                        Processing mayfly (framework: arduino; platform: atmelavr; board: mayfly)
                                                        —————————————————————————————————————————————————————–
                                                        LibraryManager: Installing id=1648 @ =0.15.3
                                                        Error: Could not find a version that satisfies the requirement ‘=0.15.3’ for your system ‘windows_amd64’
                                                        The terminal process terminated with exit code: 1

                                                      • 2018-10-11 at 9:18 PM #12602
                                                        fisherba
                                                        Participant

                                                          Neil, for the develop branch I would point to a specific commit. The current commit in develop tonight is located here:
                                                          https://github.com/EnviroDIY/ModularSensors/commit/8f1d8b58c2b9813484feed77a6074f215ecbc3e3

                                                          For my platformio.ini file’s lib deps, I would point to with the syntax:
                                                          https://github.com/EnviroDIY/ModularSensors.git#8f1d8b58c2b9813484feed77a6074f215ecbc3e3

                                                          The latest release that PlatformIO can recognize is v0.12.2, which refers to the Master branch. I see that your 0.15.3 is the current Develop branch, but that won’t be recognizable by PlatformIO until it is merged to Master.

                                                        • 2018-10-12 at 5:11 PM #12603
                                                          neilh20
                                                          Participant

                                                            Thankyou @fisherba thankyou!!!. I got the Xbee SB6 wifi communicating to data.envirodiy.org

                                                            I’ve cloned locally github.com/EnviroDIY/ModularSensors
                                                            then set to develop
                                                            and copied
                                                            enviroDIY\ModularSensors\examples\logging_to_EnviroDIY
                                                            Arduino\env02\nanolevel

                                                            Starting up platformIO I opened the folder Arduino\env02\nanolevel
                                                            in platformio.ini I set
                                                            [platformio]
                                                            ;src_dir = logging_to_EnviroDIY
                                                            src_dir = .

                                                            [env:mayfly]

                                                            lib_deps =
                                                            https://github.com/EnviroDIY/ModularSensors#8f1d8b58c2b9813484feed77a6074f215ecbc3e3
                                                            https://github.com/vshymanskyy/StreamDebugger
                                                            https://github.com/PaulStoffregen/AltSoftSerial.git
                                                            https://github.com/EnviroDIY/SoftwaterSerial_ExternalInts.git

                                                            For my logging_to_EnviroDIY.ino
                                                            I merged in specific updates for only Keller Nano
                                                            I have the XBEE cmd stream showing with new StreamDebugger

                                                            The commit – https://github.com/EnviroDIY/ModularSensors/commit/8f1d8b58c2b9813484feed77a6074f215ecbc3e3 seems to be there already, but I’ll do a manual compare.
                                                            I use meldmerge.org – fantastic compare and merge tool.
                                                            Thankyou @fisherba & @srgdamiano for the pointers – there are a lot of changes in ‘develop’ – I can’t imagine how to track them.
                                                            Amazing work @srgdamiano for bringing it all together
                                                            I’ll let my mayfly+xbee run on my desk and see what happens.

                                                          • 2018-11-12 at 5:39 PM #12649
                                                            neilh20
                                                            Participant

                                                            • 2018-11-15 at 6:27 PM #12653
                                                              neilh20
                                                              Participant

                                                                Thanks for all the help in diving into the ModularSensors code.
                                                                I’ve captured my notes on how to work as a developer with the layers of ModularSensors/TinGSM/KellerLib/Arduino code – https://github.com/neilh10/ModularSensors/wiki/git-cmds-for-preparing-for-a-PR This is really my cheat sheet – but available if anybody else wants it.

                                                            • Author
                                                              Posts
                                                            Viewing 28 reply threads
                                                            • You must be logged in to reply to this topic.