[neilh20]

@zeke-holloman – I have no idea why my previous post came out so badly, and it won’t allow me to edit it!!. My usual way of recovering strange posts.

It should be interesting to see what you get with your cell modem.

Using your sketch, which references the latest ModularSensors

Using ModularSensors Library version 0.30.0
TinyGSM Library version 0.11.4

My cell modem which is running on Verizon wouldn’t connect.  The modem issues are  technical and I believe understood, and have been discussed.

This specific modem has been working no problem in my fork  which has modifications for managing the startup of the cell modem and works for the cell company I’ve been using …. my fork currently is at ModularSensors 0.28.5

I did try compiling your program in my fork, but there is some changes in 0.30.0 that aren’t backward compatible, so I need to upgrade before I can try this.

Still with your software, referncing MS 0.30.0  I switched the modem to Digi WiFi  S6B, which I use a lot in testing,  and it worked no problem.

Once connected to an TCP/IP link – whether over cell or wifi – the passage of data is the same, though Modem AT cmds may be a little different.

So this is what I get

Sending data to [ 0 ] mqtt.thingspeak.com 5 fields will be sent to ThingSpeak <--ThingSpeakPublisher Topic [ 38 ]: channels/5940/publish/DHCKWX0HZQLHGIAX <--ThingSpeakPublisher Dumping the TX Buffer <--dataPublisherBase Message [ 106 ]: created_at=2021-07-29T18:34:00-05:00&field1=-9999.0&field2=-9999.00&field3=-9999.0&field4=4.761&field5=-25 <--ThingSpeakPublisher Opening MQTT Connection <--ThingSpeakPublisher +++OK ATLAmqtt.thingspeak.com 34.194.225.123 ATIP 1 ATDL 132.163.97.6 ATDL34.194.225.123 OK ATDE 25 ATDE75b OK ATWR OK ATAC OK ATCN OK $MQTTÂMSMSZ0G6QS4IXEHKG0PK MQTT connected after 838 ms <--ThingSpeakPublisher 0&channels/5940/publish/DHCKWX0HZQLHGIAXcreated_at=2021-07-29T18:34:00-05:00&field1=-9999.0&field2=-9999.00&field3=-9999.0&field4=4.761&field5=-25ThingSpeak topic published! Current state: 0: MQTT_CONNECTED Disconnecting from MQTT <--ThingSpeakPublisher à+++OK ATTM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

Sending data to [ 0 ] mqtt.thingspeak.com 5 fields will be sent to ThingSpeak <--ThingSpeakPublisher Topic [ 38 ]: channels/5940/publish/DHCKWX0HZQLHGIAX <--ThingSpeakPublisher Dumping the TX Buffer <--dataPublisherBase Message [ 106 ]: created_at=2021-07-29T18:34:00-05:00&field1=-9999.0&field2=-9999.00&field3=-9999.0&field4=4.761&field5=-25 <--ThingSpeakPublisher Opening MQTT Connection <--ThingSpeakPublisher +++OK ATLAmqtt.thingspeak.com 34.194.225.123 ATIP 1 ATDL 132.163.97.6 ATDL34.194.225.123 OK ATDE 25 ATDE75b OK ATWR OK ATAC OK ATCN OK $MQTTÂMSMSZ0G6QS4IXEHKG0PK MQTT connected after 838 ms <--ThingSpeakPublisher 0&channels/5940/publish/DHCKWX0HZQLHGIAXcreated_at=2021-07-29T18:34:00-05:00&field1=-9999.0&field2=-9999.00&field3=-9999.0&field4=4.761&field5=-25ThingSpeak topic published!  Current state: 0: MQTT_CONNECTED Disconnecting from MQTT <--ThingSpeakPublisher à+++OK ATTM

[neilh20]

@zeke-holloman   interesting, seems like it should work.

When I have strange issues, I delete the .pio/…   and force it to  do a refresh with all the remote libs.  I see you are using the DecagonCTD which recently changed to hydros21, but seems like it should work.

I’ve taken your platformio.ini and program  and adding my thingspeak keys, and will try running. I have a test system  with Digi LTE I’m in the process of switching so I can try your setup

In the meantime

platformio.ini to

build_flags =    -DSTREAMDEBUGGER_DBG   #usually add at end of list

lib_deps =
https://github.com/vshymanskyy/StreamDebugger  ;Debug when needed

logging_to_thinkgspeak :  look for DigiXBeeCellularTransparent  and make changes

// Create the modem object
<span style=”color: #ff0000;”>#if defined STREAMDEBUGGER_DBG</span>
<span style=”color: #ff0000;”>#include <StreamDebugger.h></span>
<span style=”color: #ff0000;”>StreamDebugger modemDebugger(modemSerial, STANDARD_SERIAL_OUTPUT);</span>
<span style=”color: #ff0000;”>#define modemSerHw modemDebugger</span>
<span style=”color: #ff0000;”>#else</span>
<span style=”color: #ff0000;”>#define modemSerHw modemSerial</span>
<span style=”color: #ff0000;”>#endif  // STREAMDEBUGGER_DBG</span>
DigiXBeeCellularTransparent modemXBCT(&<span style=”color: #ff0000;”>modemSerHw</span>, modemVccPin, modemStatusPin,
useCTSforStatus, modemResetPin,
modemSleepRqPin, apn);

[neilh20]

@zeke-holloman  a correction on my part, when I looked through the thingspeak API, both the historical and the MQTT method use the POST to deliver it to thingspeak, however they have a slightly different API.

Initially when looking at your posting I assumed you had removed the keys as it was public, however looking at your your trace it is saying that it doesn’t have the keys in it.

Your response is
Topic [ 41 ]: channels/thingSpeakChannelID/publish/thingSpeakChannelKey

and it should look something like

Topic [ 38 ]: channels/5940/publish/DHCKWX0HZQLXGIAX <–ThingSpeakPublisher

Please confirm that your run has the API keys in.

If that doesn’t work, can you post your platformio.ini

I’m committed to something for the next couple of hours but I can add describe the STREAMDEBUG setup after looking at your platformio.ini

 

 

[neilh20]

Hi @zeke-holloman – looks like the ThingSpeak keys and channel need to be added.

Add them to (the following are examples)
const char* thingSpeakMQTTKey =
“Z0G6QX4IXEHKG0PK”; // Your MQTT API Key from Account > MyProfile.
const char* thingSpeakChannelID =
“5940”; // The numeric channel id for your channel eg
const char* thingSpeakChannelKey =
“DHCKWX0HZQLXGIAX”; // The Write API Key for your channel

Your response is
Topic [ 41 ]: channels/thingSpeakChannelID/publish/thingSpeakChannelKey

and it should look something like

Topic [ 38 ]: channels/5940/publish/DHCKWX0HZQLXGIAX <–ThingSpeakPublisher

Followed by this
Dumping the TX Buffer <–dataPublisherBase
Message [ 129 ]: created_at=2021-07-28T19:04:00-08:00&field1=1&field2=2000.000&field3=-0.223&field4=4.1748&field5=4.215&field6=22.20&field7=0.0088 <–ThingSpeakPublisher
Opening MQTT Connection <–ThingSpeakPublisher
MQTT connected after 874 ms <–ThingSpeakPublisher
ThingSpeak topic published! Current state: 0: MQTT_CONNECTED
Disconnecting from MQTT <–ThingSpeakPublisher
Disconnected after 418 ms <–ThingSpeakPublisher

[neilh20]

I’ll have to get about streamdebugger as heading out the office, back in 2hrs

For debug you should see the following in platformIO.ini – add it to the end. It will force a complete recompile

<!–more–>
[env:mayfly]
monitor_speed = 115200
board = mayfly
platform = atmelavr
framework = arduino
lib_ldf_mode = deep+
lib_ignore =
RTCZero

build_flags =
-DSDI12_EXTERNAL_PCINT
-DNEOSWSERIAL_EXTERNAL_PCINT
-DMQTT_MAX_PACKET_SIZE=240
-DTINY_GSM_RX_BUFFER=64
-DTINY_GSM_YIELD_MS=2
-DMS_DATAPUBLISHERBASE_DEBUG
-DMS_THINGSPEAKPUBLISHER_DEBUG

[neilh20]

Hi @zeke – good check by Rick.

Looking at your error, it does say connection error so implies it didn’t even connect to thingspeak.

I checked this 5months ago and it worked for me. What is your channel number?. This allows a view of if any data has arrived. The channel I used for testing, a bit simplistic  https://thingspeak.com/channels/5940

You could also enable some debug and share it.

For platformIO this is
<div>
<div>build_flags =</div>
</div>
<div>
<div>    -DMS_DATAPUBLISHERBASE_DEBUG</div>
<div>    -DMS_THINGSPEAKPUBLISHER_DEBUG</div>
</div>
 

[neilh20]

Hi @zeke thanks for the qu.  I’ve got the thingspeak working as a POST, but don’t regularly test it. I believe the MQTT is a different route, and I personally haven’t used it.

For the Digi Corp LTE Cat-M1 cell modems, I do ensure that they are upgraded to the latest version with XCTU & XCTU board. It is a bit painful, but worked for me. If there is any problem with the Digi LTE modem I force a reset to defaults.

The provisioning, customizations all go in the code ~ hologram as you have it, and then in the class should have there is  mqtt.thingspeak.com . ThingSpeakPublisher.cpp

I’ll check my notes to see what I did.
If nothing else, I would enable the modem stream to see what its doing. Look for

// Use this to create a modem if you want to monitor modem communication through
// a secondary Arduino stream.  Make sure you install the StreamDebugger
// library! https://github.com/vshymanskyy/StreamDebugger
#if defined STREAMDEBUGGER_DBG

I’ll check out my notes to see what exactly I did on testing.

[neilh20]

Its often seems simple, but the practical side of it can be complex – like if certificates are needed.  AWS can scale, but that requires a lot of layers. As I understand it AWS dices and slices the options and the bills are a big surprise.

In software design the question that comes up is, what’s the functionality that is desired? How do you break the functionality into small layers.  What currently works, and what is needed.

Does it have to be AWS or can it be something else?

https://thingspeak.com/  Can do data collection, and then my experience is the API can pull it off pretty easily.  I did this with an experimental  http://azonde.org/pm2/LiveRainGauge3.htm

The code is in the browser, not that I really follow how it all works!!  and also here https://github.com/neilh10/thingspeakmultichannel-highcharts

The thingspeak costs are pretty well defined.

For the Mayfly which ever way you look at data collection, then reliably pushing data to the endpoint is also needed, which  I’ve got in my sensors, and its on my list to PR but haven’t got there yet.  https://github.com/EnviroDIY/ModularSensors/issues/194

 

[neilh20]

@dan-wachusett  I’ve used WOFpy but there are some issues. https://github.com/ODM2/ODM2DataSharingPortal/issues/445

https://github.com/ODM2/ODM2DataSharingPortal/issues/450

 

Is that the same function as pulling with a GET, or does the GET add extra value – possibly my ignorance in not understanding different methods.

[neilh20]

I know my local Sonoma State University is against devices connecting to the network. They even gone as far as removing all  ethernet sockets. As part of the local engineering capstone project using LoRa the students went off campus for testing, as the LoRa gateway needed to connect to a WiFi network

The simple answer for WiFi is what Sara described.

Because of the wider security concerns for IP based systems, there is a whole lot of technical initiatives  to manage admittance to larger local networks – so security is likely to get more proscribed.

Just an FYI, even if you got WiFi connecting, I’m still having reliability issues with the Modular Sensors S6B WiFi interface. https://github.com/EnviroDIY/ModularSensors/issues/347

 

[Author]

Posts