Welcome to EnviroDIY, a community for do-it-yourself environmental science and monitoring. EnviroDIY is part of WikiWatershed, an initiative of Stroud Water Research Center designed to help people advance knowledge and stewardship of fresh water.
New to EnviroDIY? Start here

neilh20

Forum Replies Created

Viewing 10 posts - 141 through 150 (of 371 total)
1 2 3 … 14 15 16 … 36 37 38
  • Author
    Posts
  • 2021-12-28 at 3:30 PM in reply to: Stability Testing ~ how to do it? #16216
    neilh20
    Participant

      In the early part of this year 2021, I did some regression testing using an Insitu LT500/SDI12 with a modbus board and telecom LTE, and I ran into a hard to find issue on the Mayfly 0.5b  – the Uart Rx line is unterminated..

      When extending features, its not unusual that problems are uncovered in new ways of exercising the software.

      As part of a terminal based set the date on a simple Mayfly, I introduced a command line interface.

      The objective was to be able to set the Date and Time on a Mayfly when its been installed or about to be installed. Typically a connected Mayfly gets the date when it polls NIST or by special date setting program.

      A command line can be very useful in other ways as I brought out here

      https://www.envirodiy.org/topic/how-to-dump-contents-of-file-on-sd-card-to-serial/#post-15830

      For the initial command line interface I used String.

      Now, it turns out that String has some down sides, and its valuable to understand these downsides – so as not to trip over them like I did.

      https://cpp4arduino.com/2018/11/06/what-is-heap-fragmentation.html

      https://cpp4arduino.com/2018/11/21/eight-tips-to-use-the-string-class-efficiently.html

       

      So I created a potential issue in the software, that for incoming characters for the UART RX, I added them to a String. The hardware unfortunately can generate a lot of random characters, and this can cause the String to use up a lot of memory.

      In the process of debugging this I wrote a utility to dump the ram and showing how much stack and heap used I called it dumpFreeRam() . This gives a periodic visual of the ram allocation, and what might be using it.

      The critical issue is monitoring during integration testing how much free ram is available, and what might be using it.

      I did find one bug this way when a lot of ram got used up with the above usage of String. I’ve changed it to a more traditional fast circular buffer.

      This utility at the end of each invocation calculates a summary “Free ram never allocated”. However this by itself doesn’t indicate when the ram actually got allocated, so the debug listing needs to be evaluated for what event caused any ram to be used.

      The example below shows that a program used up to 273 bytes after running for a couple of hours. 6161-591=273bytes. Fortunately  longer testing doesn’t see more ram leakage.

      [2021-08-01 16:22:14.881] Free ram never allocated between (bytes dec) 6144 and 6161

      and after two hours

      [2021-08-01 18:20:15.013] Free ram never allocated between (bytes dec) 5888 and 5912

       

      The output on the terminal using Teraterm looks like this (using a Nanolevel/RS485)

      after running for two hours

       

      The utility is in my fork, search for dumpFreeRam() in following

      https://github.com/neilh10/ModularSensors/commit/842eab880fdf6548d8b6d14951e4f0ad45727ecd

      2021-12-22 at 4:22 PM in reply to: MMW timeouts #16196
      neilh20
      Participant

        @aufdenkampe many thanks. Appreciate this is covering timing issues that are a bit hard to flush out.

        I suppose I’m the messenger in checking the system to what appears to be the bar from the Mayfly, but its a pretty hard area to checkout – all I can say is that as the messenger I’ve seen the challenges in other systems and was trying to give the headsup with figuring out what the best way of characterizing 5the system with https://github.com/ODM2/ODM2DataSharingPortal/issues/524

        My LTE systems when responding is now doing so in 10seconds, or timing out at 25seconds.
        info on https://github.com/ODM2/ODM2DataSharingPortal/issues/542
        What’s nice it does appear to be getting to the database.

        2021-12-10 at 3:38 PM in reply to: Status update on MMW? #16171
        neilh20
        Participant

          I cleared browsing date for “cached images and files” & “browsing history” (a complete clear of everything is quite educational to recover from)
          https://support.google.com/accounts/answer/32050
          and it worked. thanks for the tip

          2021-12-10 at 1:51 PM in reply to: Status update on MMW? #16168
          neilh20
          Participant

            Thanks for the update. The system that I had that wasn’t reporting this morning is now reporting – https://monitormywatershed.org/sites/TUCA_PO03/ as of Dec. 10, 2021, 10:30 a.m. (UTC-08:00)

            I’m logging in using FireFox for my production sites and Chrome for my test sites.
            For the above it looks different under the different browsers, and the TSV is accessible under FireFox, but not under Chrome, do you want me to log an issue on it?

            2021-12-09 at 1:05 PM in reply to: Status update on MMW? #16161
            neilh20
            Participant

              For my status, there is one site that started uploading, OK – which is good as its about 3hours drive time.

              One site that hasn’t updated. Its on private land and requires permission to enter the property, and its usually only done when there is a group of activities on the site. Its about one hour drive away.
              Both are on verizon.

              For my local test system, running over verizon I monitor the output of the connection process. Its having a lot of timeouts. The timeout is set to 5seconds.

              2021-12-08 at 1:03 PM in reply to: Status update on MMW? #16141
              neilh20
              Participant

                I’m seeing the same – and strange data losses been going on since 2021-12-03 20:15 PST,
                I’ve entered it as
                https://github.com/ODM2/ODM2DataSharingPortal/issues/535

                I also asked three weeks ago how to do a test for a reliable data server
                https://github.com/ODM2/ODM2DataSharingPortal/issues/524

                and over a year ago, I asked for what could a Reliable Delivery model algorithm look like
                https://github.com/ODM2/ODM2DataSharingPortal/issues/485

                2021-12-07 at 5:45 PM in reply to: Data not appearing in sparklines or on Time Series Analyst #16137
                neilh20
                Participant

                  Interesting, thanks for the post. I was wondering I’m not getting data posted from my units.
                  However the data posted can be read, so seems only the backend has failed.
                  The last readings are Dec. 7, 2021, 9:45 a.m. (UTC-08:00)

                  2021-12-07 at 2:37 PM in reply to: Data not appearing in sparklines or on Time Series Analyst #16135
                  neilh20
                  Participant

                    Hey wow good to see. I’ve been experimenting on the staging version of this, and so the new TSA is much faster and slightly different, but soooo much better.!!.

                    What I’ve seen is initially the new graphs draw a years worth of data and auto-range.
                    It can be narrowed with the “Date Range” by clicking on Begin – and selecting a new start date. Then clicking Update Plot.
                    New sensors can be added through the “Add series” – with gorgeous colours.
                    Zooming in on data can be performed with visual selection – that is click on graph, hold mouse button down and move cursor, then release cursor

                    2021-11-30 at 5:21 PM in reply to: USB to UART Mayfly, Serial Ports Not Connecting #16128
                    neilh20
                    Participant

                      Thanks for the link –
                      Just helped two people/students connect their windows machines to the Mayfly USB-C CP210x
                      – both required the installation of the “CP210x Windows Drivers”
                      from https://www.silabs.com/developers/usb-to-uart-bridge-vcp-drivers then Downloads

                      2021-11-15 at 7:13 PM in reply to: Clock Sync for Mayfly v1.0 #16105
                      neilh20
                      Participant

                        Oh interesting I hadn’t seen the PYsync

                        In EnviroDIY_DS3231\examples\PCsync\PYsync.py

                        There is a line looking for the port
                        # Keep only ports with the device communication type
                        device_ports = [
                        p for p in ports
                        if ‘VID_0403+PID_6001’ in p[2]
                        ]

                        From device manager, PORTS , Silicon Labs CP210x USB to UART
                        In Events
                        I see
                        Device USB\VID_10C4&PID_EA60\0001 requires further installation.

                        Don’t know whether that will work

                      • Author
                        Posts
                      Viewing 10 posts - 141 through 150 (of 371 total)
                      1 2 3 … 14 15 16 … 36 37 38