Activity

I cleared browsing date for “cached images and files” & “browsing history” (a complete clear of everything is quite educational to recover from)
https://support.google.com/accounts/answer/32050
and it worked. thanks for the tip

@neilh, do a hard reset of your browser cache our URL to get TSV working on Chrome. That is definitely the issue there. We’ll be adding “cache-busting” code in our next update.

@aufdenkampe – I’ve confirmed that indeed, all stations are now reporting data as expected.

Thanks for the update. The system that I had that wasn’t reporting this morning is now reporting – https://monitormywatershed.org/sites/TUCA_PO03/ as of Dec. 10, 2021, 10:30 a.m. (UTC-08:00)

I’m logging in using FireFox for my production sites and Chrome for my test sites.
For the above it looks different under the different browsers, and the…[Read more]

We started the proxy forwarding about 45 minutes ago, so all data should now be rerouted from our old servers to the new servers on AWS.

Let us know how things look!

Thanks @aufdenkampe, that sounds like a great plan. I like the idea of the forwarding log too.

We’ll watch for a change in stations’ status once the proxy is in place.

@mbarney and others,

After doing some research, we decided that shutting down our old servers for 30 minutes would not likely force the DNS on devices to refresh.

Our plan is to set up proxy forwarding on our LimnoTech server to instantly forward all data to the AWS servers. We plan to do that today, and will also set up a forwarding log to…[Read more]

Interesting – we have 1 site with the new LTEbee, and it also weathered the cutover successfully.

@aufdenkampe, As of now, we’re not seeing any of our Digi stations that are able to upload to the new server as a result of the temporary shutdown of your old server, assuming that that indeed took place yesterday.

Matt

All of our stations that are using our new sim7089 EnviroDIY LTEbee made the changeover automatically.  The majority of our stations that still have the Digi Xbee LTE module have been offline since the server change, however a few random ones seemed to be fine and were online immediately after the changeover.  The rest of the stations with Digi b…[Read more]

For my status, there is one site that started uploading, OK – which is good as its about 3hours drive time.

One site that hasn’t updated. Its on private land and requires permission to enter the property, and its usually only done when there is a group of activities on the site. Its about one hour drive away.
Both are on verizon.

For my local…[Read more]

@mbarny, that is very helpful to know that you have had success with power-cycling the stations. We’re seeing a few other sites switch over on their own in the last 24 hours.

We have 1 site that resumed uploading data at 12/8/2021 14:45 ET, with no known intervention.

5 sites are uploading after we restarted them, and 18 sites that have not reported since the cutover to the new server.

@mbarney, wouldn’t that have been great if the Hologram pause worked! Thanks for trying it out!

Have you seen any of your sites start reporting again in the last 24 hours? The couple that I’ve been tracking have not, which is disappointing. I definitely recognize how painful it would be to manually power cycle these stations.

We’ll likely try to…[Read more]

We tried the Hologram SIM pause idea on one station, but it had no effect, which isn’t too surprising.

@aufdenkampe, please keep us informed about the timing of the temporary shutdown of the old server. Thanks for your efforts.

Matt

Thanks @heather, and @aufdenkampe, that does makes sense, with the symptoms we’re seeing. Good to know that the data are not being lost in the interim.

Shutting down the old server temporarily sounds like a good idea; all of our devices are on a 15-minute sampling period, so 20+ minutes should cause at least one failed message from the device,…[Read more]

Fortunately, we deployed to AWS region US-east-2 (Ohio) and it was the US-east-1 (Virgina) data center that went down, so our release yesterday was not affected by the AWS outage!

Unfortunately, we’ve seen two issues with caching mess up our otherwise smooth release:

1. Domain Name Service (DNS) caches are causing some devices to send data to our…

[Read more]