Home › Forums › Mayfly Data Logger › Connecting SIM7080 to New Thingspeak
- This topic has 7 replies, 4 voices, and was last updated 2022-01-12 at 2:03 PM by Zeke Holloman.
-
AuthorPosts
-
-
2022-01-05 at 4:33 PM #16225
Hi there! I recently received an email from Thingspeak stating that they are planning on shutting down their legacy Thingspeak server this month. I’m working to convert my Thingspeak channels over to from mqtt.thingspeak.com to mqtt3.thingspeak.com.
I’m using a Mayfly v1.0 alongside you all’s SIM7080 card. It publishs data to legacy ThingSpeak with no problems! I figured it would be just as simple as changing the website URL to “mqtt3.thingspeak.com”, but it’s proving more difficult than that. Have you all had any experience with this? Any suggestions are appreciated.
Thank you and have a good one!
-
2022-01-05 at 6:18 PM #16226
Can you provide more details especially in reference to your code and how you are connecting to thingspeak?
-
2022-01-05 at 9:44 PM #16229
Sorry, I coded up the ThingSpeak publisher ages ago only to prove to myself that I could get an MQTT publisher working. I’ve never used it beyond that. I don’t have a paid account so after the trial period I was locked out and I’m not willing to do much testing on an account tied to my personal email.
Anyway, the code for ThingSpeak is in the ThingSpeak publisher in the publishers folder. It’s using PubSubClient under the hood. The outgoing message is formatted based on whatever the required structure was at the time. If that’s changed, then the whole formatting step will need to be rewritten.
I wouldn’t guess the restructuring of the json or whatever ThingSpeak wants would be too hard, but I don’t have time to look at it right now. If ThingSpeak now requires a secure connection (SSL, etc) or if they more want more metadata then there might need to be more tweaking for that.
-
2022-01-06 at 3:16 PM #16237
@zeke-holloman I agree with @srgdamiano, Mathworks have gone down the path of making it difficult to test a small number of devices against thingspeak, since it was added to ModularSensors publishers.
A bit of shame since I’ve had some simple devices working reliably against it since 2013.
Also unfortunately, “the real world” is that testing is against a matrix – release ModularSensors xx.xx against thingspeak – so I think you are on the leading edge for ModularSensors Publisher v mqtt3.thingspeak 🙂
Taking a scan of their mqtt3, it seems like they are suggesting you create new devices when moving to their mqtt3 – so at a GUESS, that suggests they have a whole new server architecture and subscription management. So be great if you share results – stuff that doesn’t work is as valuable as stuff that does work.
When looking at a new service, its also useful to consider what it takes to prove how well it works and how far you want to go.
Typically new development goes in stages a) protoyping b) integration testing c) reliability/stability testing.
MMW has a lot of capability, and very mature, but in the “reliability testing” of Mayfly 0.5 and Digi LTE CAT-M1 modems, from a debug analysis, there is a possible timing problem with the tinyGSM device driver https://github.com/EnviroDIY/ModularSensors/issues/396 – which suggests that if the MMW servers response where a lot faster, faster than the current fastest 1.2seconds, it might break the current deployments with Digi LTE-CAT-M1 XB3-C-A2-UT-001. Its what happens in testing sometimes.
I’ve had very good experience so far with “reliability/stability testing” of ModularSensors 0.32.2/(Mayfly 0.5+ Digi LTE-CAT-M1), but the server response through the cellphone network is pretty slow, at its fastest 1.2seconds.
For mqtt3.thingspeak you are back to “a)prototyping”.
For another wireless protocol LoRa/Thingsnetwork, with max payload of 51bytes, which seems unlikely that it could work with MMW API, I’m checking out Arduino.cc mqtt/IoT or Adafruit mqtt as having plans for smaller devices. So I would describe I’m trying to figure how to prototype that interface.
I should note – https://github.com/EnviroDIY/ModularSensors/releases/tag/v0.32.2 doesn’t identify what features work with new Mayfly 1.0A3 boards. The modem SIM7080 , isn’t identified as being supported yet, or tested with ModularSensors – though as its open source we can all see what’s in the software. I guess, user test at their own risk.
I have only done minimal testing on Mayfly 1.0A3, as its typical for organizations to announce what they support and regression testing. My testing, is with a student capstone project as a standalone sensor so far, though they should get to cellular LTE integration soon. On paper analysis I’ve identified a potential comms issues with Mayfly1.0A3 – see https://github.com/EnviroDIY/EnviroDIY_Mayfly_Logger/issues
I’m making ModularSensors detect in runtime whether its Mayfly 0.5 or 1.0 – so the user/program doesn’t have to compile for one or the other.
-
2022-01-07 at 5:03 PM #16244
Hi all! Thank you for all of your feedback and tips! I spent some time this week looking at the mqtt3.thingspeak.com server and I was able to get a channel up and running on it.
Overall, it’s not drastically different than mqtt.thingspeak.com. The main aspect being removed in the legacy Thingspeak server is the API keys. Thingspeak now relies on the Channel ID, the client ID, the username, and the password to send data to a specific channel.
There are also a few formatting changes for sending the data, which can be found on this site, under July 2021:
I did end up editing the ThingSpeakPublisher library in order to get the data to upload correctly, but I don’t think I had to change the data type.
Here’s the Arduino code:
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433/** =========================================================================* @file DRWI_sim7080LTE.ino* @brief Example for DRWI CitSci LTE sites.** This example shows proper settings for the following configuration:** Mayfly v1.0 board* EnviroDIY sim7080 LTE module (with Hologram SIM card)* Hydros21 CTD sensor* Campbell Scientific OBS3+ Turbidity sensor** @author Sara Geleskie Damiano <sdamiano@stroudcenter.org>* @copyright (c) 2017-2021 Stroud Water Research Center (SWRC)* and the EnviroDIY Development Team* This example is published under the BSD-3 license.** Hardware Platform: EnviroDIY Mayfly Arduino Datalogger** DISCLAIMER:* THIS CODE IS PROVIDED "AS IS" - NO WARRANTY IS GIVEN.* ======================================================================= */// ==========================================================================// Defines for the Arduino IDE// NOTE: These are ONLY needed to compile with the Arduino IDE.// If you use PlatformIO, you should set these build flags in your// platformio.ini// ==========================================================================/** Start [defines] */#ifndef TINY_GSM_RX_BUFFER#define TINY_GSM_RX_BUFFER 64#endif#ifndef TINY_GSM_YIELD_MS#define TINY_GSM_YIELD_MS 2#endif#ifndef MQTT_MAX_PACKET_SIZE#define MQTT_MAX_PACKET_SIZE 256#endif/** End [defines] */// ==========================================================================// Include the libraries required for any data logger// ==========================================================================/** Start [includes] */// The Arduino library is needed for every Arduino program.#include <Arduino.h>// EnableInterrupt is used by ModularSensors for external and pin change// interrupts and must be explicitly included in the main program.#include <EnableInterrupt.h>// Include the main header for ModularSensors#include <ModularSensors.h>/** End [includes] */// ==========================================================================// Data Logging Options// ==========================================================================/** Start [logging_options] */// The name of this program fileconst char* sketchName = "thingspeaksingular_mayfly_1point0_GW1.ino";// Logger ID, also becomes the prefix for the name of the data file on SD cardconst char* LoggerID = "logger";// How frequently (in minutes) to log dataconst uint8_t loggingInterval = 1;// Your logger's timezone.const int8_t timeZone = -5; // Eastern Standard Time// NOTE: Daylight savings time will not be applied! Please use standard time!// Set the input and output pins for the logger// NOTE: Use -1 for pins that do not applyconst int32_t serialBaud = 115200; // Baud rate for debuggingconst int8_t greenLED = 8; // Pin for the green LEDconst int8_t redLED = 9; // Pin for the red LEDconst int8_t buttonPin = 21; // Pin for debugging mode (ie, button pin)const int8_t wakePin = 31; // MCU interrupt/alarm pin to wake from sleep// Mayfly 0.x D31 = A7// Set the wake pin to -1 if you do not want the main processor to sleep.// In a SAMD system where you are using the built-in rtc, set wakePin to 1const int8_t sdCardPwrPin = -1; // MCU SD card power pinconst int8_t sdCardSSPin = 12; // SD card chip select/slave select pinconst int8_t sensorPowerPin = 22; // MCU pin controlling main sensor power/** End [logging_options] */// ==========================================================================// Wifi/Cellular Modem Options// ==========================================================================/** Start [sim7080] */// For almost anything based on the SIMCom SIM7080G#include <modems/SIMComSIM7080.h>#define modemSerial Serial1// NOTE: Extra hardware and software serial ports are created in the "Settings// for Additional Serial Ports" sectionconst int32_t modemBaud = 9600; // SIM7080 does auto-bauding by default, but for simplicity we set to 9600// Modem Pins - Describe the physical pin connection of your modem to your board// NOTE: Use -1 for pins that do not applyconst int8_t modemVccPin = 18; // MCU pin controlling modem power --- Pin 18 is the power enable pin for the bee socket on Mayfly v1.0,// use -1 if using Mayfly 0.5b or if the bee socket is constantly powered (ie you changed SJ18 on Mayfly1.0 to 3.3v)const int8_t modemStatusPin = 19; // MCU pin used to read modem statusconst int8_t modemSleepRqPin = 23; // MCU pin for modem sleep/wake requestconst int8_t modemLEDPin = redLED; // MCU pin connected an LED to show modem// status// Network connection informationconst char* apn = "hologram"; // APN connection name, typically Hologram unless you have a different provider's SIM card. Change as needed// Create the modem objectSIMComSIM7080 modem7080(&modemSerial, modemVccPin, modemStatusPin,modemSleepRqPin, apn);// Create an extra reference to the modem by a generic nameSIMComSIM7080 modem = modem7080;/** End [sim7080] */// ==========================================================================// ==========================================================================// Using the Processor as a Sensor// ==========================================================================/** Start [processor_sensor] */#include <sensors/ProcessorStats.h>// Create the main processor chip "sensor" - for general metadataconst char* mcuBoardVersion = "v0.5b"; // is only used for onboard battery voltage calculation,// "v0.5b" is safe to use on Mayfly v1.0 boards because the formula is the sameProcessorStats mcuBoard(mcuBoardVersion);/** End [processor_sensor] */// ==========================================================================// Maxim DS3231 RTC (Real Time Clock)// ==========================================================================/** Start [ds3231] */#include <sensors/MaximDS3231.h>// Create a DS3231 sensor objectMaximDS3231 ds3231(1);/** End [ds3231] */// ==========================================================================// Meter Hydros 21 Conductivity, Temperature, and Depth Sensor// ==========================================================================/** Start [hydros21] */#include <sensors/MeterHydros21.h>const char* hydrosSDI12address = "1"; // The SDI-12 Address of the Hydros 21const uint8_t hydrosNumberReadings = 6; // The number of readings to averageconst int8_t SDI12Power = sensorPowerPin; // Power pin (-1 if unconnected)const int8_t SDI12Data = 7; // The SDI12 data pin// Create a Meter Hydros 21 sensor objectMeterHydros21 hydros(*hydrosSDI12address, SDI12Power, SDI12Data,hydrosNumberReadings);/** End [hydros21] */Variable *Hydros21_TempF = new MeterHydros21_Temp(&hydros, "12345678-abcd-1234-ef00-1234567890ab");//Hydros temperature in degrees Fahrenheitfloat calculateHydros21degF(void) {float MeterHydros21_TempC = Hydros21_TempF->getValue();float MeterHydros21_TempF = MeterHydros21_TempC*(1.8)+32;if (MeterHydros21_TempC == -9999) {MeterHydros21_TempF = -9999;}return MeterHydros21_TempF;}const char* Hydros21TempFVarName = "Temperature";const char *Hydros21TempVarFUnit = "Degree";int Hydros21TempFVarResolution = 2;const char *Hydros21TempFUUID = "12345678-abcd-1234-ef00-1234567890ab";const char *Hydros21TempFVarCode = "Hydros21TempF";Variable *HydrosCalcTempF = new Variable(calculateHydros21degF, Hydros21TempFVarResolution,Hydros21TempFVarName, Hydros21TempVarFUnit, Hydros21TempFVarCode, Hydros21TempFUUID);Variable *Hydros21_Depthft = new MeterHydros21_Depth(&hydros, "12345678-abcd-1234-ef00-1234567890ab");//Hydros temperature in degrees Fahrenheitfloat calculateHydros21feet(void) {float MeterHydros21_Depth_mm = Hydros21_Depthft->getValue();float MeterHydros21_Depth_ft = MeterHydros21_Depth_mm/304.8;if (MeterHydros21_Depth_mm == -9999) {MeterHydros21_Depth_ft = -9999;}return MeterHydros21_Depth_ft;}const char* Hydros21DepthftVarName = "Depth";const char *Hydros21DepthftVarUnit = "Feet";int Hydros21DepthftVarResolution = 2;const char *Hydros21DepthftUUID = "12345678-abcd-1234-ef00-1234567890ab";const char *Hydros21DepthftVarCode = "Hydros21Depthft";Variable *HydrosCalcDepthft = new Variable(calculateHydros21feet, Hydros21DepthftVarResolution,Hydros21DepthftVarName, Hydros21DepthftVarUnit, Hydros21DepthftVarCode, Hydros21DepthftUUID);// ==========================================================================// Campbell OBS 3 / OBS 3+ Analog Turbidity Sensor// ==========================================================================/** Start [obs3] */// ==========================================================================// Creating the Variable Array[s] and Filling with Variable Objects// ==========================================================================/** Start [variable_arrays] */Variable* variableList[] = {new MeterHydros21_Cond(&hydros),HydrosCalcTempF,HydrosCalcDepthft,new ProcessorStats_Battery(&mcuBoard),};// All UUID's, device registration, and sampling feature information can be pasted directly from Monitor My Watershed.//To get the list, click the "View token UUID list" button on the upper right of the site page.// *** CAUTION --- CAUTION --- CAUTION --- CAUTION --- CAUTION ***// Check the order of your variables in the variable list!!!// Be VERY certain that they match the order of your UUID's!// Rearrange the variables in the variable list ABOVE if necessary to match!// Do not change the order of the variables in the section below.// *** CAUTION --- CAUTION --- CAUTION --- CAUTION --- CAUTION ***// Replace all of the text in the following section with the UUID array from MonitorMyWatershed// --------------------- Beginning of Token UUID List ---------------------------------------// Count up the number of pointers in the arrayint variableCount = sizeof(variableList) / sizeof(variableList[0]);// Create the VariableArray objectVariableArray varArray;/** End [variable_arrays] */// ==========================================================================// The Logger Object[s]// ==========================================================================/** Start [loggers] */// Create a logger instanceLogger dataLogger(LoggerID, loggingInterval, &varArray);/** End [loggers] */// ==========================================================================// Creating Data Publisher[s]// ==========================================================================// Create a channel with fields on ThingSpeak in advance// The fields will be sent in exactly the order they are in the variable array.// Any custom name or identifier given to the field on ThingSpeak is irrelevant.// No more than 8 fields of data can go to any one channel. Any fields beyond// the eighth in the array will be ignored.//Will be removed in a future iteration of thingspeakconst char* thingSpeakChannelID ="INPUT_CHANNEL_ID_HERE"; // The numeric channel id for your channelconst char* thingSpeakMQTTusername ="INPUT_USERNAME_HERE"; //Your MQTT usernameconst char* thingSpeakMQTTpassword ="INPUT_PASSWORD_HERE"; //Your MQTT passwordconst char* thingSpeakMQTTclientID ="INPUT_CLIENT_ID_HERE"; //Your MQTT client ID// Create a data publisher for ThingSpeak#include <publishers/ThingSpeakPublisher.h>ThingSpeakPublisher TsMqtt;/** End [loggers] */// ==========================================================================// Working Functions// ==========================================================================/** Start [working_functions] */// Flashes the LED's on the primary boardvoid greenredflash(uint8_t numFlash = 4, uint8_t rate = 75) {for (uint8_t i = 0; i < numFlash; i++) {digitalWrite(greenLED, HIGH);digitalWrite(redLED, LOW);delay(rate);digitalWrite(greenLED, LOW);digitalWrite(redLED, HIGH);delay(rate);}digitalWrite(redLED, LOW);}// Reads the battery voltage// NOTE: This will actually return the battery level from the previous update!float getBatteryVoltage() {if (mcuBoard.sensorValues[0] == -9999) mcuBoard.update();return mcuBoard.sensorValues[0];}// ==========================================================================// Arduino Setup Function// ==========================================================================/** Start [setup] */void setup() {// Start the primary serial connectionSerial.begin(serialBaud);// Print a start-up note to the first serial portSerial.print(F("Now running "));Serial.print(sketchName);Serial.print(F(" on Logger "));Serial.println(LoggerID);Serial.println();Serial.print(F("Using ModularSensors Library version "));Serial.println(MODULAR_SENSORS_VERSION);Serial.print(F("TinyGSM Library version "));Serial.println(TINYGSM_VERSION);Serial.println();// Start the serial connection with the modemmodemSerial.begin(modemBaud);// Set up pins for the LED'spinMode(greenLED, OUTPUT);digitalWrite(greenLED, LOW);pinMode(redLED, OUTPUT);digitalWrite(redLED, LOW);// Blink the LEDs to show the board is on and starting upgreenredflash();pinMode(20, OUTPUT); //for proper operation of the onboard flash memory chip's ChipSelect (Mayfly v1.0 and later)// Set the timezones for the logger/data and the RTC// Logging in the given time zoneLogger::setLoggerTimeZone(timeZone);// It is STRONGLY RECOMMENDED that you set the RTC to be in UTC (UTC+0)Logger::setRTCTimeZone(0);// Attach the modem and information pins to the loggerdataLogger.attachModem(modem);modem.setModemLED(modemLEDPin);dataLogger.setLoggerPins(wakePin, sdCardSSPin, sdCardPwrPin, buttonPin,greenLED);// Begin the variable array[s], logger[s], and publisher[s]varArray.begin(variableCount, variableList);dataLogger.begin(LoggerID, loggingInterval, &varArray);TsMqtt.begin(dataLogger, &modem.gsmClient,thingSpeakChannelID,thingSpeakMQTTusername,thingSpeakMQTTpassword,thingSpeakMQTTclientID);// Note: Please change these battery voltages to match your battery// Set up the sensors, except at lowest battery levelif (getBatteryVoltage() > 3.4) {Serial.println(F("Setting up sensors..."));varArray.setupSensors();}/** Start [setup_sim7080] */modem.setModemWakeLevel(HIGH); // ModuleFun Bee inverts the signalmodem.setModemResetLevel(HIGH); // ModuleFun Bee inverts the signalSerial.println(F("Waking modem and setting Cellular Carrier Options..."));modem.modemWake(); // NOTE: This will also set up the modemmodem.gsmModem.setBaud(modemBaud); // Make sure we're *NOT* auto-bauding!modem.gsmModem.setNetworkMode(38); // set to LTE only// 2 Automatic// 13 GSM only// 38 LTE only// 51 GSM and LTE onlymodem.gsmModem.setPreferredMode(1); // set to CAT-M// 1 CAT-M// 2 NB-IoT// 3 CAT-M and NB-IoT/** End [setup_sim7080] */// Sync the clock if it isn't valid or we have battery to spareif (getBatteryVoltage() > 3.55 || !dataLogger.isRTCSane()) {// Synchronize the RTC with NIST// This will also set up the modemdataLogger.syncRTC();}// Create the log file, adding the default header to it// Do this last so we have the best chance of getting the time correct and// all sensor names correct// Writing to the SD card can be power intensive, so if we're skipping// the sensor setup we'll skip this too.if (getBatteryVoltage() > 3.4) {Serial.println(F("Setting up file on SD card"));dataLogger.turnOnSDcard(true); // true = wait for card to settle after power updataLogger.createLogFile(true); // true = write a new headerdataLogger.turnOffSDcard(true); // true = wait for internal housekeeping after write}// Call the processor sleepSerial.println(F("Putting processor to sleep\n"));dataLogger.systemSleep();}/** End [setup] */// ==========================================================================// Arduino Loop Function// ==========================================================================/** Start [loop] */// Use this short loop for simple data logging and sendingvoid loop() {// Note: Please change these battery voltages to match your battery// At very low battery, just go back to sleepif (getBatteryVoltage() < 3.4) {dataLogger.systemSleep();}// At moderate voltage, log data but don't send it over the modemelse if (getBatteryVoltage() < 3.55) {dataLogger.logData();}// If the battery is good, send the data to the worldelse {dataLogger.logDataAndPublish();}}/** End [loop] */ -
2022-01-07 at 5:11 PM #16245123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230/*** @file ThingSpeakPublisher.cpp* @copyright 2020 Stroud Water Research Center* Part of the EnviroDIY ModularSensors library for Arduino* @author Sara Geleskie Damiano <sdamiano@stroudcenter.org>** @brief Implements the ThingSpeakPublisher class.*/#include "ThingSpeakPublisher.h"// ============================================================================// Functions for the EnviroDIY data portal receivers.// ============================================================================// Constant values for MQTT publish// I want to refer to these more than once while ensuring there is only one copy// in memoryconst char* ThingSpeakPublisher::mqttServer = "mqtt3.thingspeak.com";const int ThingSpeakPublisher::mqttPort = 1883;const char* ThingSpeakPublisher::mqttClientID = THING_SPEAK_CLIENT_ID;const char* ThingSpeakPublisher::mqttUser = THING_SPEAK_USER_NAME;const char* ThingSpeakPublisher::mqttPass = THING_SPEAK_PASS_WORD;// ConstructorsThingSpeakPublisher::ThingSpeakPublisher() : dataPublisher() {// MS_DBG(F("ThingSpeakPublisher object created"));_thingSpeakChannelID = NULL;_thingSpeakUsername = NULL;_thingSpeakPassword = NULL;_thingSpeakClientID = NULL;}ThingSpeakPublisher::ThingSpeakPublisher(Logger& baseLogger, uint8_t sendEveryX,uint8_t sendOffset): dataPublisher(baseLogger, sendEveryX, sendOffset) {// MS_DBG(F("ThingSpeakPublisher object created"));_thingSpeakChannelID = NULL;_thingSpeakUsername = NULL;_thingSpeakPassword = NULL;_thingSpeakClientID = NULL;}ThingSpeakPublisher::ThingSpeakPublisher(Logger& baseLogger, Client* inClient,uint8_t sendEveryX, uint8_t sendOffset): dataPublisher(baseLogger, inClient, sendEveryX, sendOffset) {// MS_DBG(F("ThingSpeakPublisher object created"));_thingSpeakChannelID = NULL;_thingSpeakUsername = NULL;_thingSpeakPassword = NULL;_thingSpeakClientID = NULL;}ThingSpeakPublisher::ThingSpeakPublisher(Logger& baseLogger,const char* thingSpeakChannelID,const char* thingSpeakUsername,const char* thingSpeakPassword,const char* thingSpeakClientID,uint8_t sendEveryX, uint8_t sendOffset): dataPublisher(baseLogger, sendEveryX, sendOffset) {setChannelID(thingSpeakChannelID);setUsername(thingSpeakUsername);setPassword(thingSpeakPassword);setClientID(thingSpeakClientID);// MS_DBG(F("ThingSpeakPublisher object created"));}ThingSpeakPublisher::ThingSpeakPublisher(Logger& baseLogger, Client* inClient,const char* thingSpeakChannelID,const char* thingSpeakUsername,const char* thingSpeakPassword,const char* thingSpeakClientID,uint8_t sendEveryX, uint8_t sendOffset): dataPublisher(baseLogger, inClient, sendEveryX, sendOffset) {setChannelID(thingSpeakChannelID);setUsername(thingSpeakUsername);setPassword(thingSpeakPassword);setClientID(thingSpeakClientID);// MS_DBG(F("ThingSpeakPublisher object created"));}// DestructorThingSpeakPublisher::~ThingSpeakPublisher() {}void ThingSpeakPublisher::setChannelID(const char* thingSpeakChannelID) {_thingSpeakChannelID = thingSpeakChannelID;// MS_DBG(F("Channel ID set!"));}void ThingSpeakPublisher::setUsername(const char* thingSpeakUsername) {_thingSpeakUsername = thingSpeakUsername;// MS_DBG(F("Username set!"));}void ThingSpeakPublisher::setPassword(const char* thingSpeakPassword) {_thingSpeakPassword = thingSpeakPassword;// MS_DBG(F("Password set!"));}void ThingSpeakPublisher::setClientID(const char* thingSpeakClientID) {_thingSpeakClientID = thingSpeakClientID;// MS_DBG(F("Client ID set!"));}// Sets all 3 ThingSpeak parametersvoid ThingSpeakPublisher::setThingSpeakParams(const char* channelID,const char* Username,const char* Password,const char* ClientID) {setChannelID(channelID);setUsername(Username);setPassword(Password);setClientID(ClientID);}// A way to begin with everything already setvoid ThingSpeakPublisher::begin(Logger& baseLogger, Client* inClient,const char* thingSpeakChannelID,const char* thingSpeakUsername,const char* thingSpeakPassword,const char* thingSpeakClientID) {setChannelID(thingSpeakChannelID);setUsername(thingSpeakUsername);setPassword(thingSpeakPassword);setClientID(thingSpeakClientID);dataPublisher::begin(baseLogger, inClient);}void ThingSpeakPublisher::begin(Logger& baseLogger,const char* thingSpeakChannelID,const char* thingSpeakUsername,const char* thingSpeakPassword,const char* thingSpeakClientID) {setChannelID(thingSpeakChannelID);setUsername(thingSpeakUsername);setPassword(thingSpeakPassword);setClientID(thingSpeakClientID);dataPublisher::begin(baseLogger);}// This sends the data to ThingSpeak// bool ThingSpeakPublisher::mqttThingSpeak(void)int16_t ThingSpeakPublisher::publishData(Client* outClient) {bool retVal = false;// Make sure we don't have too many fields// A channel can have a max of 8 fieldsif (_baseLogger->getArrayVarCount() > 8) {MS_DBG(F("No more than 8 fields of data can be sent to a single ""ThingSpeak channel!"));MS_DBG(F("Only the first 8 fields worth of data will be sent."));}uint8_t numChannels = min(_baseLogger->getArrayVarCount(), 8);MS_DBG(numChannels, F("fields will be sent to ThingSpeak"));// Create a buffer for the portions of the request and responsechar tempBuffer[26] = "";char topicBuffer[25] = "channels/"; //was 42strcat(topicBuffer, _thingSpeakChannelID);strcat(topicBuffer, "/publish");//MS_DBG(F("Topic ["), strlen(topicBuffer), F("]:"), String(topicBuffer));emptyTxBuffer();_baseLogger->formatDateTime_ISO8601(Logger::markedEpochTime).toCharArray(tempBuffer, 26);strcat(txBuffer, "#P");strcat(txBuffer, "created_at=");strcat(txBuffer, tempBuffer);txBuffer[strlen(txBuffer)] = '&';for (uint8_t i = 0; i < numChannels; i++) {strcat(txBuffer, "field");itoa(i + 1, tempBuffer, 10); // BASE 10strcat(txBuffer, tempBuffer);txBuffer[strlen(txBuffer)] = '=';_baseLogger->getValueStringAtI(i).toCharArray(tempBuffer, 26);strcat(txBuffer, tempBuffer);if (i + 1 != numChannels) { txBuffer[strlen(txBuffer)] = '&'; }}//strcat(txBuffer, "&status=MQTTPUBLISH");MS_DBG(F("Message ["), strlen(txBuffer), F("]:"), String(txBuffer));// Set the client connection parameters_mqttClient.setClient(*outClient);_mqttClient.setServer(mqttServer, mqttPort);// Make sure any previous TCP connections are closed// NOTE: The PubSubClient library used for MQTT connect assumes that as// long as the client is connected, it must be connected to the right place.// Closing any stray client sockets here ensures that a new client socket// is opened to the right place.// client is connected when a different socket is openif (outClient->connected()) { outClient->stop(); }// Make the MQTT connection// Note: the client id and the user name do not mean anything for// ThingSpeakMS_DBG(F("Opening MQTT Connection"));MS_START_DEBUG_TIMER;if (_mqttClient.connect(_thingSpeakClientID, _thingSpeakUsername, _thingSpeakPassword)) {MS_DBG(F("MQTT connected after"), MS_PRINT_DEBUG_TIMER, F("ms"));if (_mqttClient.publish(topicBuffer, txBuffer)) {PRINTOUT(F("ThingSpeak topic published! Current state:"),parseMQTTState(_mqttClient.state()));retVal = true;} else {PRINTOUT(F("MQTT publish failed with state:"),parseMQTTState(_mqttClient.state()));retVal = false;}} else {PRINTOUT(F("MQTT connection failed with state:"),parseMQTTState(_mqttClient.state()));delay(1000);retVal = false;}// Disconnect from MQTTMS_DBG(F("Disconnecting from MQTT"));MS_RESET_DEBUG_TIMER_mqttClient.disconnect();MS_DBG(F("Disconnected after"), MS_PRINT_DEBUG_TIMER, F("ms"));return retVal;}
^^ Here’s the edited cpp for the publisher
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228/*** @file ThingSpeakPublisher.h* @copyright 2020 Stroud Water Research Center* Part of the EnviroDIY ModularSensors library for Arduino* @author Sara Geleskie Damiano <sdamiano@stroudcenter.org>** @brief Contains the ThingSpeakPublisher subclass of dataPublisher for* publishing data to ThingSpeak using the MQTT protocol.*/// Header Guards#ifndef SRC_PUBLISHERS_THINGSPEAKPUBLISHER_H_#define SRC_PUBLISHERS_THINGSPEAKPUBLISHER_H_// Debugging Statement// #define MS_THINGSPEAKPUBLISHER_DEBUG#ifdef MS_THINGSPEAKPUBLISHER_DEBUG#define MS_DEBUGGING_STD "ThingSpeakPublisher"#endif/*** @brief The MQTT User Name** @note The user name doesn't actually mean anything for ThingSpeak*///edited, defines username, password, and client ID#define THING_SPEAK_USER_NAME ""#define THING_SPEAK_PASS_WORD ""#define THING_SPEAK_CLIENT_ID ""//edited for mqtt3//#define USESECUREMQTT// Included Dependencies#include "ModSensorDebugger.h"#undef MS_DEBUGGING_STD#include "dataPublisherBase.h"#include <PubSubClient.h>// ============================================================================// Functions for the EnviroDIY data portal receivers.// ============================================================================/*** @brief The ThingSpeakPublisher subclass of dataPublisher for publishing data* to ThingSpeak using the MQTT protocol.** When sending data to ThingSpeak the order of the variables in the variable* array attached to your logger is __crucial__. The results from the variables* in the VariableArray will be sent to ThingSpeak in the order they are in the* array; that is, the first variable in the array will be sent as Field1, the* second as Field2, etc. Any UUID's or custom variable codes are ignored for* ThingSpeak. They will only appear in the header of your file on the SD card.* Giving a variable a custom variable code like "Field3" will **NOT** make that* variable field 3 on ThingSpeak. The third variable in the array will always* be "Field3". Any text names you have given to your fields in ThingSpeak are* also irrelevant.** @ingroup the_publishers*/class ThingSpeakPublisher : public dataPublisher {public:// Constructors/*** @brief Construct a new ThingSpeak Publisher object with no members* initialized.*/ThingSpeakPublisher();/*** @brief Construct a new ThingSpeak Publisher object** @note If a client is never specified, the publisher will attempt to* create and use a client on a LoggerModem instance tied to the attached* logger.** @param baseLogger The logger supplying the data to be published* @param sendEveryX Currently unimplemented, intended for future use to* enable caching and bulk publishing* @param sendOffset Currently unimplemented, intended for future use to* enable publishing data at a time slightly delayed from when it is* collected** @note It is possible (though very unlikey) that using this constructor* could cause errors if the compiler attempts to initialize the publisher* instance before the logger instance. If you suspect you are seeing that* issue, use the null constructor and a populated begin(...) within your* set-up function.*/explicit ThingSpeakPublisher(Logger& baseLogger, uint8_t sendEveryX = 1,uint8_t sendOffset = 0);/*** @brief Construct a new ThingSpeak Publisher object** @param baseLogger The logger supplying the data to be published* @param inClient An Arduino client instance to use to print data to.* Allows the use of any type of client and multiple clients tied to a* single TinyGSM modem instance* @param sendEveryX Currently unimplemented, intended for future use to* enable caching and bulk publishing* @param sendOffset Currently unimplemented, intended for future use to* enable publishing data at a time slightly delayed from when it is* collected** @note It is possible (though very unlikey) that using this constructor* could cause errors if the compiler attempts to initialize the publisher* instance before the logger instance. If you suspect you are seeing that* issue, use the null constructor and a populated begin(...) within your* set-up function.*/ThingSpeakPublisher(Logger& baseLogger, Client* inClient,uint8_t sendEveryX = 1, uint8_t sendOffset = 0);/*** @brief Construct a new ThingSpeak Publisher object** @param baseLogger The logger supplying the data to be published* @param thingSpeakChannelID The numeric channel id for your channel* @param sendEveryX Currently unimplemented, intended for future use to* enable caching and bulk publishing* @param sendOffset Currently unimplemented, intended for future use to* enable publishing data at a time slightly delayed from when it is*/ThingSpeakPublisher(Logger& baseLogger,const char* thingSpeakChannelID,const char* thingSpeakUsername,const char* thingSpeakPassword,const char* thingSpeakClientID,uint8_t sendEveryX = 1, uint8_t sendOffset = 0);/*** @brief Construct a new ThingSpeak Publisher object** @param baseLogger The logger supplying the data to be published* @param inClient An Arduino client instance to use to print data to.* Allows the use of any type of client and multiple clients tied to a* single TinyGSM modem instance* @param thingSpeakChannelID The numeric channel id for your channel* @param sendEveryX Currently unimplemented, intended for future use to* enable caching and bulk publishing* @param sendOffset Currently unimplemented, intended for future use to* enable publishing data at a time slightly delayed from when it is* collected*/ThingSpeakPublisher(Logger& baseLogger, Client* inClient,const char* thingSpeakChannelID,const char* thingSpeakUsername,const char* thingSpeakPassword,const char* thingSpeakClientID,uint8_t sendEveryX = 1, uint8_t sendOffset = 0);/*** @brief Destroy the ThingSpeak Publisher object*/virtual ~ThingSpeakPublisher();// Returns the data destinationString getEndpoint(void) override {return String(mqttServer);}/*** @brief Set the ThingSpeak channel ID** @param thingSpeakChannelID The numeric channel id for your channel*/void setChannelID(const char* thingSpeakChannelID);//editedvoid setUsername(const char* thingSpeakUsername);void setPassword(const char* thingSpeakPassword);void setClientID(const char* thingSpeakClientID);void setThingSpeakParams(const char* channelID,const char* Username,const char* Password, const char* ClientID);// A way to begin with everything already set/*** @copydoc dataPublisher::begin(Logger& baseLogger, Client* inClient)* @param thingSpeakChannelID The numeric channel id for your channel*/void begin(Logger& baseLogger, Client* inClient,const char* thingSpeakChannelID,const char* thingSpeakUsername,const char* thingSpeakPassword,const char* thingSpeakClientID);/*** @copydoc dataPublisher::begin(Logger& baseLogger)* @param thingSpeakChannelID The numeric channel id for your channel*/void begin(Logger& baseLogger,const char* thingSpeakChannelID,const char* thingSpeakUsername,const char* thingSpeakPassword,const char* thingSpeakClientID);// This sends the data to ThingSpeak// bool mqttThingSpeak(void);int16_t publishData(Client* outClient) override;protected:/*** @anchor ts_mqqt_vars* @name Portions of the MQTT data publication** @{*/static const char* mqttServer; ///< The MQTT serverstatic const int mqttPort; ///< The MQTT portstatic const char* mqttClientName; ///< The MQTT client namestatic const char* mqttChannelID; ///< The MQTT channel IDstatic const char* mqttUser; ///< The MQTT user namestatic const char* mqttPass; ///< The MQTT passwordstatic const char* mqttClientID;/**@}*/private:// Keys for ThingSpeakconst char* _thingSpeakChannelID;const char* _thingSpeakUsername;const char* _thingSpeakPassword;const char* _thingSpeakClientID;PubSubClient _mqttClient;};#endif // SRC_PUBLISHERS_THINGSPEAKPUBLISHER_H_^^ Here’s the edited code for the header file.
-
2022-01-07 at 7:47 PM #16247
@zeke-holloman hey congrats,!!! Thanks for the posting and analysis, I could also build from that.
Wow… the power of open source that you can dive in and tweak it, so you can make it work. Go @zeke-holloman !!!
Just an FYI, what I do to facilitate easy maintenance, and easy future merges, I rename my customizations to a “unique name space” eg ThingSpeakPublisher3a.h/.cpp and then also the classes ThingSpeakPublisher3a.
Then if you come to do an update “git pull” and there are updates in the main line “master” ThinkSpeakPublisher.h/cpp you’ll have a smoother ride with being able to track the updates, and either adopt them or stay with your customizations.
-
2022-01-12 at 2:03 PM #16252
@neilh20 Hey, thanks! WOW! That’s awesome it worked for you as well. Hope it ends up being helpful!
I will say, it wasn’t a quick process. It was a lot of jumping back and forth between ThingSpeak documentation and code, but I was SUPER excited when it finally uploaded.
Aaaah, ok, that makes sense. Thank you!
-
-
AuthorPosts
- You must be logged in to reply to this topic.