Welcome to EnviroDIY, a community for do-it-yourself environmental science and monitoring. EnviroDIY is part of WikiWatershed, an initiative of Stroud Water Research Center designed to help people advance knowledge and stewardship of fresh water.
New to EnviroDIY? Start here

neilh20

Forum Replies Created

Viewing 10 posts - 141 through 150 (of 376 total)
1 2 3 … 14 15 16 … 36 37 38
  • Author
    Posts
  • 2022-01-15 at 3:52 PM in reply to: Deep Cycle 12v Battery Recommendations #16269
    neilh20
    Participant

      edit to the above email:  Sorry I don’t have any experience with the 12V batteries, or 12V to 5V.

      I would note that the Mayfly1.0 charger is different than 0.5b, and I’m seeing that it wants a slightly higher+5V than the Mayfly 0.5b for it to do a 0.5A charge of the LiIon battery. I am using USB charging monitor “USB Meter” to monitor the charge rate.

      I definitely agree with @ckillen  that whatever methods’ are used, some monitoring is required to figure out where the power goes and is it working as expected

      2022-01-14 at 10:03 PM in reply to: Deep Cycle 12v Battery Recommendations #16267
      neilh20
      Participant

        I’m also deeply into power analysis as the way of keeping everything going. I was working with some engineering students and got them to do a spreadsheet, which I modified and added and can be used to build a budget for the power that you think you need, and what the power consumers might be.  I then look for the major user of the power, and let the project be driven by that. ie if its a sensor that needs 12V and consumes a lot of power then you need to have the large battery.

        I also reduce the power by changing how often it turns on the cell modem. ie from 15minutes to 60minutes. The project with students we are only talking about turning it on every 24 hours. However this is a lot of software that I’ve added to my fork.

        I also have a battery threshold that I stop any transmission, and then when there is enough solar power harvested start transmitting the stored readings and then the new readings. Its not trivial software to do this, but it has now been well tested.

        Happy to share more about power savings if its of interest.

        One idea that I’ve seen with the Mayfly 1.0 and the new power routing, is that there can be the  LiIon battery plugged in, with solar panel, and then a top up USB Battery pack plugged in.

        However I do have to say that the Mayfly 0.5 and also 1.0 battery measurement is challenging, and I’ve tried to phrase it here. IHMO its a really basic issue, and has caused me no end of technical headaches. Mayfly 1.0A3 is more of problem than 0.5  https://github.com/EnviroDIY/EnviroDIY_Mayfly_Logger/issues/32

        Attachments:
        1. Power-Consumption2-enviroDIY.xlsx
        2022-01-07 at 7:47 PM in reply to: Connecting SIM7080 to New Thingspeak #16247
        neilh20
        Participant

          @zeke-holloman  hey congrats,!!! Thanks for the posting and analysis, I could also build from that.

          Wow… the power of open source that you can dive in and tweak it, so you can make it work. Go @zeke-holloman   !!!

          Just an FYI, what I do to facilitate easy maintenance, and easy future merges, I rename my customizations to a “unique name space”  eg ThingSpeakPublisher3a.h/.cpp and then also the classes ThingSpeakPublisher3a.

          Then if you come to do an update “git pull” and there are updates in the main line “master” ThinkSpeakPublisher.h/cpp you’ll have a smoother ride  with being able to track the updates,  and either adopt them or stay with your customizations.

          2022-01-07 at 3:40 PM in reply to: Compiling for Mayfly v0.5 or v1.0 #16242
          neilh20
          Participant

            Hi Matt

            The short answer :),  there is a difference in hardware ports, and that can impact the interface to the modem.  I expect to support the Mayfly 0.5b for the foreseeable future for devices in the field, and the Mayfly 1.0x features are in evolution. I appreciate the support and context of SWRC, hardware is available for purchase, software is gittable, ~ however there has been no description of regression testing or any visible plan for the Mayfly1.0 features. (The MMW has a visible plan https://github.com/ODM2/ODM2DataSharingPortal/milestones)

            Sounds like you have put the software together and its worked. Probably its the default configuration of the ports. Whew!.

            Looking at the schematics, the Mayfly1.0A3 v Mayfly0.5b ports are different when it comes to the modem. For Mayfly 1.0 I appreciate the better control of the modem subsystem – I so like to be able to ensure a known configuration of a complex module (the secret; periodic reset) – just to avoid long term reliability problems I’ve seen in other modules.

            If there is a Mayfly 1.0 specific hardware integration issue, or usage of its new features,  its going to need a specific workaround, and only used if the Mayfly 1.0 is operating.

            The long answer, is I’m working towards simplifying the complex configuration of the “logger platform”. The goal I have is that the “logger platform” should be configuration driven – that is configured by reading what revision the Mayfly board is at, and also in a uSD:ms_cfg.ini file for the network/UUIDs, and in the future configure for the modem, and configure for sensors.

            I  have the infrastructure already built to program the onboard EEPROM for revision info, so relatively easy to read it out.

            By far the biggest challenge to any software platform with 24/7 criteria is stability – embedded platform stability is even more challenging.  I put together this discussion – as much about having a whole system testable – ModularSensors/Mayfly +MMW – https://github.com/ODM2/ODM2DataSharingPortal/issues/524 which is also dependent on https://github.com/ODM2/ODM2DataSharingPortal/issues/485

            Stability needs to be characterized and that is white box testing. Typically for a saleable product with software on hardware, its a core customer expectation (sales contract). Simplify all the complexity, to core stability confidence.

            The target for engineering, can you change one item ~ software or hardware, and still have some confidence in the whole “logger platform” stability. The poster child for this is the Digi WiFi S6, which was working (in 2019 I think) – I put a system in the field with an external WiFi antenna + WiFi gateway and it worked (then the system got stolen). I used the WiFi module as a proving ground for stability of the general Mayfly when the LTE was still evolving and also for basic interfaces to the server. I designed the core ReliableDelivery algorithms over WiFi, as the data plans where pretty expensive at that point. Then something changed, probably the tinyGsm lib, and I wasn’t watching it as I had moved to using LTE and SDI-12, and getting them stable/working. The WiFi S6 was broken for probably 6months (and still is on main release). When I got the WiFI S6 back working middle of last year, it required a fork and rework of the tinyGsm drivers.  All of this is in my private fork, as to be honest stability characterization is hard/complicated. None of the known problems are reflected in the ModularSensors modems page. (https://envirodiy.github.io/ModularSensors/index.html#mainpage_modems) . What of course would be nice is what was the revision it was tested against – MS version, module hw/sw, and known problems in later releases.

            Hardware also has stability characterization process – Mayfly SDI-12 implementation is a subset of https://sdi-12.org/specification  – Mayfly 1.0A3 was perhaps trying for +12V generation required by the specification. Mayfly1.0A3 still claims that it can do +12V generation (with no power spec), but my versions haven’t worked with real world requirement,  https://github.com/EnviroDIY/EnviroDIY_Mayfly_Logger/issues/33

            The history I’ve found, with other projects and also the Mayfly, you need to deal with the real world device foibles (and availability). Last time I looked at Xbee WiFi,  the Digi WiFi module was the only one with a u.fl connector for an external antenna – critical for better range. So i’ve stuck with keeping the Digi WiFI module.

            Others may have something similar – but how about an objective to make it easy to change – switch one sensor LT500/SDI-12 to  CTD/SDI-12

            My perspective for an embedded “logger platform”, is it has a defined configuration to work and be deployed – Mayfly0.5b+carrier board Digi LTE-CATM1 modem+ antenna  +4400mA battery + 3W solar panel + RS485board(for SDI12 +12V) +sensor LT500.

            That configuration needs to work. If some of the components are switched out – eg to Mayfly 1.0 + Digi LTE-CATM1(less carrier board)  – I would also like it to work in the same way. However Mayfly 1.0 ports interface to the Dig LTE-CATM1 differently.

            So hope you don’t mind the long answer, back to reading the hardware configuration,  so for software stability testing, when its loaded with a defined .hex, I’d like to be easily adjust for hardware features!! …. Mayfly 0.5b, Mayfly 1.0A3, … Mayfly 1.x

            2022-01-06 at 3:16 PM in reply to: Connecting SIM7080 to New Thingspeak #16237
            neilh20
            Participant

              @zeke-holloman I agree with @srgdamiano,  Mathworks have gone down the path of making it difficult to test a small number of devices against thingspeak, since it was added to ModularSensors publishers.

              A bit of shame since I’ve had some simple devices working reliably against it since 2013.

              Also unfortunately, “the real world” is that testing is against a matrix – release ModularSensors xx.xx against thingspeak – so I think you are on the leading edge for ModularSensors Publisher v mqtt3.thingspeak  🙂

              Taking a scan of their  mqtt3,  it seems like they are suggesting you create new devices when moving to their mqtt3  – so at a GUESS, that suggests they have a whole new server architecture and subscription management. So be great if you share results – stuff that doesn’t work is as valuable as stuff that does work.

              When looking at a new service, its also useful to consider what it takes to prove how well it works  and how far you want to go.

              Typically new development goes in stages a) protoyping b) integration testing c) reliability/stability testing.

              MMW has a lot of capability, and very mature,  but in the “reliability testing” of  Mayfly 0.5 and Digi LTE CAT-M1 modems, from a debug analysis, there is a possible timing  problem with the tinyGSM device driver  https://github.com/EnviroDIY/ModularSensors/issues/396 – which suggests that if the MMW servers response where a lot faster, faster than the current fastest 1.2seconds, it might break the current deployments with Digi LTE-CAT-M1 XB3-C-A2-UT-001.   Its what happens in testing sometimes.

              I’ve had very good experience so far with “reliability/stability testing” of ModularSensors 0.32.2/(Mayfly 0.5+ Digi LTE-CAT-M1), but the server response through the cellphone network is pretty slow,  at its fastest 1.2seconds.

              For mqtt3.thingspeak you are back to “a)prototyping”.

              For  another wireless protocol LoRa/Thingsnetwork,  with max payload of 51bytes, which seems unlikely that it could work with MMW API, I’m checking out Arduino.cc mqtt/IoT or Adafruit mqtt as having plans  for smaller devices.  So I would describe I’m trying to figure how to prototype that interface.

              I should note – https://github.com/EnviroDIY/ModularSensors/releases/tag/v0.32.2 doesn’t identify what features work with new Mayfly 1.0A3 boards. The modem SIM7080 , isn’t identified as being supported yet, or tested with ModularSensors – though as its open source we can all see what’s in the software. I guess, user test at their own risk.

              I have only done minimal testing on Mayfly 1.0A3, as its typical for organizations to announce what they support and regression testing. My testing, is with a student capstone project as a standalone sensor so far, though they should get to cellular LTE integration soon.  On paper analysis I’ve identified a potential comms issues with Mayfly1.0A3  – see https://github.com/EnviroDIY/EnviroDIY_Mayfly_Logger/issues

              I’m making ModularSensors detect in runtime whether its Mayfly 0.5 or 1.0 – so the user/program doesn’t have to compile for one or the other.

              2021-12-28 at 3:30 PM in reply to: Stability Testing ~ how to do it? #16216
              neilh20
              Participant

                In the early part of this year 2021, I did some regression testing using an Insitu LT500/SDI12 with a modbus board and telecom LTE, and I ran into a hard to find issue on the Mayfly 0.5b  – the Uart Rx line is unterminated..

                When extending features, its not unusual that problems are uncovered in new ways of exercising the software.

                As part of a terminal based set the date on a simple Mayfly, I introduced a command line interface.

                The objective was to be able to set the Date and Time on a Mayfly when its been installed or about to be installed. Typically a connected Mayfly gets the date when it polls NIST or by special date setting program.

                A command line can be very useful in other ways as I brought out here

                https://www.envirodiy.org/topic/how-to-dump-contents-of-file-on-sd-card-to-serial/#post-15830

                For the initial command line interface I used String.

                Now, it turns out that String has some down sides, and its valuable to understand these downsides – so as not to trip over them like I did.

                https://cpp4arduino.com/2018/11/06/what-is-heap-fragmentation.html

                https://cpp4arduino.com/2018/11/21/eight-tips-to-use-the-string-class-efficiently.html

                 

                So I created a potential issue in the software, that for incoming characters for the UART RX, I added them to a String. The hardware unfortunately can generate a lot of random characters, and this can cause the String to use up a lot of memory.

                In the process of debugging this I wrote a utility to dump the ram and showing how much stack and heap used I called it dumpFreeRam() . This gives a periodic visual of the ram allocation, and what might be using it.

                The critical issue is monitoring during integration testing how much free ram is available, and what might be using it.

                I did find one bug this way when a lot of ram got used up with the above usage of String. I’ve changed it to a more traditional fast circular buffer.

                This utility at the end of each invocation calculates a summary “Free ram never allocated”. However this by itself doesn’t indicate when the ram actually got allocated, so the debug listing needs to be evaluated for what event caused any ram to be used.

                The example below shows that a program used up to 273 bytes after running for a couple of hours. 6161-591=273bytes. Fortunately  longer testing doesn’t see more ram leakage.

                [2021-08-01 16:22:14.881] Free ram never allocated between (bytes dec) 6144 and 6161

                and after two hours

                [2021-08-01 18:20:15.013] Free ram never allocated between (bytes dec) 5888 and 5912

                 

                The output on the terminal using Teraterm looks like this (using a Nanolevel/RS485)

                after running for two hours

                 

                The utility is in my fork, search for dumpFreeRam() in following

                https://github.com/neilh10/ModularSensors/commit/842eab880fdf6548d8b6d14951e4f0ad45727ecd

                2021-12-22 at 4:22 PM in reply to: MMW timeouts #16196
                neilh20
                Participant

                  @aufdenkampe many thanks. Appreciate this is covering timing issues that are a bit hard to flush out.

                  I suppose I’m the messenger in checking the system to what appears to be the bar from the Mayfly, but its a pretty hard area to checkout – all I can say is that as the messenger I’ve seen the challenges in other systems and was trying to give the headsup with figuring out what the best way of characterizing 5the system with https://github.com/ODM2/ODM2DataSharingPortal/issues/524

                  My LTE systems when responding is now doing so in 10seconds, or timing out at 25seconds.
                  info on https://github.com/ODM2/ODM2DataSharingPortal/issues/542
                  What’s nice it does appear to be getting to the database.

                  2021-12-10 at 3:38 PM in reply to: Status update on MMW? #16171
                  neilh20
                  Participant

                    I cleared browsing date for “cached images and files” & “browsing history” (a complete clear of everything is quite educational to recover from)
                    https://support.google.com/accounts/answer/32050
                    and it worked. thanks for the tip

                    2021-12-10 at 1:51 PM in reply to: Status update on MMW? #16168
                    neilh20
                    Participant

                      Thanks for the update. The system that I had that wasn’t reporting this morning is now reporting – https://monitormywatershed.org/sites/TUCA_PO03/ as of Dec. 10, 2021, 10:30 a.m. (UTC-08:00)

                      I’m logging in using FireFox for my production sites and Chrome for my test sites.
                      For the above it looks different under the different browsers, and the TSV is accessible under FireFox, but not under Chrome, do you want me to log an issue on it?

                      2021-12-09 at 1:05 PM in reply to: Status update on MMW? #16161
                      neilh20
                      Participant

                        For my status, there is one site that started uploading, OK – which is good as its about 3hours drive time.

                        One site that hasn’t updated. Its on private land and requires permission to enter the property, and its usually only done when there is a group of activities on the site. Its about one hour drive away.
                        Both are on verizon.

                        For my local test system, running over verizon I monitor the output of the connection process. Its having a lot of timeouts. The timeout is set to 5seconds.

                      • Author
                        Posts
                      Viewing 10 posts - 141 through 150 (of 376 total)
                      1 2 3 … 14 15 16 … 36 37 38