I needed a data logger which could take a stream of serial data and record it onto an SD card. To do this I based the hardware on the DataDuino and changed the microcontroller code.

This is an overview of the idea and the arduino code required to implement it.

This was designed to work with the pedal power equipment I have been designing.

Concept

A number of the devices I have been creating recently stream data via a serial output. This data can then be used with other devices, such as displayed on an LED display or stored onto a computer (via a USB-serial converter). Sometimes I need a stand-alone data recorder for this serial data stream.

I created the DataDuino to record anaolgue and digital data onto an SD card. This hardware can be used to read serial data and record it to the SD card.

SD write cycles

One issue with the SD card is that it will only last for a certain number of write cycles – typically 100,000. While that seems like a lot, if data is written once per second (86400 samples per day) then an SD card will only last just a bit over 1 day. To prevent this SD cards usually have wear-levelling built in which helps to even out the number of write cycles over the whole size of the card. Writing data once per second onto a 2GB SD card typically gives a SD card life of around 100 days.

In this situation I am not recording data 24 hours a day. It will be taking data only when the pedal generator is in use. This means that I do not need to worry as much about the SD write cycles in this application.

If I was creating a data-logger which would work 24hrs a day, then I would need to think about this issue and perhaps write the data in ‘chunks’ at set intervals. This would reduce the write cycles.

Another option would be to use a much larger SD card – these have more memory therefore, with wear-levelling, the card would last longer.

Code flow overview

Rather than take analog and digital readings whenever there is a write pulse from the clock, I want the code to monitor the serial port. When serial data arrives we want to check that it is in the correct format, strip out the useful data, store it into EEPROM and then write it to the SD card in chunks when the EEPROM has filled up.

Data will be written using a version of LLAP (Lightweight Local Automation Protocol) with a unique data reference. We want to strip out the data reference, the type of data, the value of the data and also add a timestamp from the real time clock.

This unit has been designed to record from the power log unit. This outputs data at 115200 baud via the serial port. The data is streamed every second and takes the format of voltage reading, current reading , power reading.

I save all three values and store them onto the SD card. These are saved along with a timestamp. A new file is created each day.

Here are some photos of the set-up:

The DataDuino with just two input wires – ground and data (attached to Rx pin). Power was provided by batteries.

Here you can see the three LLAP messages sent every second and then the line written to the SD card.

Arduino code for the Dataduino

The Dataduino is an Arduino-based datalogger. It is available here. The following code has been written for the dataduino to save serial data onto pin 0 (the Rx pin). This code is also available via my GITHUB repository.

This code seems to work quite well and stores the serial data to the SD card. Every so often there is an issue with the voltage reading, which could be due to the device which sends out the data, rather than this serial data recorder.

Maybe this code would be useful to you? Please feel free to adapt and use for your own use. You will need to include the RTC PCF8563 library (which must be downloaded) and the SD card library (which is included within the Arduino IDE).

/********************************************************
/****** DataDuino - Arduino Serial DAQ UNIT *************
/****** by Matt Little **********************************
/****** Date: 30/10/12 **********************************
/****** info@re-innovation.co.uk ************************
/****** www.re-innovation.co.uk *************************
/********************************************************

  See www.re-innovation.co.uk for information and construction details
  This is the SERIAL data storage version.
  
  This is sample code for the DataDuino PCB.

/*************Details of Code*****************************

  The DataDuino is an Arduino based SD card datalogger.
  A PCF8563 Realt Time Clock is used to timestamp the data.
  Data is written every time there is serial data in the correct format.

  Data is written to a .csv file created on an SD card.
  A new file is created each day. If file alreay present then data is appended.
  The file name is created from the reference number and the date in the format:
  RXXDXXXX.csv, where RXX is the reference number and DXXXX is the date in the format DDMM. 

  SERIAL Commands (at 115200 baud) :
  "aXXCHID??E--"
  This will change the reference number to ??
  "a--ID?----"
  This will return the device ID 


  Updates: 
  30/10/12 Code written - Matt Little
  31/10/12 writing code description and sorting out new sample timings - Matt Little
  19/9/13  Getting code to work on Arduino 1.0.5 - Matt Little
  19/9/13  Updated the Rtc_Pcf8563 to work with Arduino 1.0.5 - Matt Little
  19/9/13  Changed getData and sortData to 'listen' for serial data - Matt Little
  19/9/13  Removed all references to temperature - not needed - Matt Little
  23/9/13  Changed format of data - Matt Little

 
 //*********SD CARD DETAILS***************************	
 The SD card circuit:
 SD card attached to SPI bus as follows:
 ** MOSI - pin 11
 ** MISO - pin 12
 ** CLK - pin 13
 ** CS - pin 10
 ** Card detect - pin 6
 
 SD card code details:
 created  24 Nov 2010
 updated 2 Dec 2010
 by Tom Igoe
 
 //************ Real Time Clock code*******************
 A PCF8563 RTC is attached to pins:
 ** A4 - SDA (serial data)
 ** A5 - SDC (serial clock)
 ** D2 - Clock out - This gives a 1 second pulse to record the data
 
 RTC PCF8563 code details:
 By Joe Robertson, jmr
 orbitalair@bellsouth.net
 
**********************************************************************************************************/


/************ External Libraries*****************************/
#include <stdlib.h>
#include <Wire.h>          // Required for RTC
#include <Rtc_Pcf8563.h>   // RTC library
#include <SD.h>            // SD card library
#include <avr/pgmspace.h>  // Library for putting data into program memory
#include <EEPROM.h>        // For writing values to the EEPROM
#include <avr/eeprom.h>

/************User variables and hardware allocation**********************************************/

/******* SD CARD*************/
const int chipSelect = 10; // The SD card Chip Select pin 10
const int cardDetect = 6;  // The SD card detect is on pin 6
// The other SD card pins (D11,D12,D13) are all set within SD.h
int cardDetectOld = LOW;  // This is the flag for the old reading of the card detect

/*************Real Time Clock*******/
Rtc_Pcf8563 rtc;
#define I2C_RTC 0x51 // 7 bit address (without last bit - look at the datasheet)
int RTCinterrupt = 0;  // RTC interrupt - This is pin 2 of ardunio - which is INT0

/********* Output LED *************/
const int LEDred = 5;  // The output led is on pin 5

/********* Digital Data Storage  ***********/
int D7 = 0;  // Digital input
int D8 = 0;  // Digital input
int D9 = 0;  // Digital input

/********** Analogue Data Storage ************/
int analog0 = 0;  // Analog input
int analog1 = 0;  // Analog input
int analog2 = 0;  // Analog input
int analog3 = 0;  // Analog input

// ****** Serial Data Read***********
// Variables for the serial data read
char inByte;         // incoming serial char
String str_buffer = "";  // This is the holder for the string which we will display

//********Variables for the Filename*******************
char filename[] = "RXXDXXXX.csv";  // This is a holder for the full file name
//int refnumber;     // The house number here, which is stored in EEPROM
char deviceID[3]; // A buffer to hold the device ID
File datafile;   // The logging file
String dataString;    // This is the holder for the data as a string. Start as blank
int counter = 0;   // Clue is in the name - its a counter.
long dataCounter = 0;  // This holds the number of seconds since the last data store
long sampleTime = 10;  // This is the time between samples for the DAQ

// Variables for the Pulse Counter
int pulseinterrupt = 1;  // Pulse Counter Interrupt - This is pin 3 of arduino - which is INT1
volatile long pulsecounter = 0;  // This counts pulses from the flow sensor  - Needs to be long to hold number
volatile long pulsecounterold = 0;  // This is storage for the old flow sensor - Needs to be long to hold number


volatile int writedataflag = HIGH;  // A flag to tell the code when to write data

// Varibales for writing to EEPROM
int hiByte;      // These are used to store longer variables into EERPRPROM
int loByte;

// These next ints are for the filename conversion

int day_int =0;      // To find the day from the Date for the filename
int day_int1 =0;
int day_int2 =0;
int month_int = 0;
int month_int1 = 0;
int month_int2 = 0;
int year_int = 0;  // Year
int hour_int = 0;
int min_int = 0;
int sec_int = 0;

//**********STRINGS TO USE****************************
String comma = ",";
String date;        // The stored date from filename creation
String newdate;     // The new date, read every time 

//String dataType;   // This holds the dataType ("P", "V", "I")
String dataVoltage;  // This holds the actual data
String dataCurrent;  // This holds the actual data
String dataPower;  // This holds the actual data

// These are Char Strings - they are stored in program memory to save space in data memory
// These are a mixutre of error messages and serial printed information
const char headers[] PROGMEM = "Ref,Date,Time,Voltage,Current,Power";  // Headers for the top of the file
const char headersOK[] PROGMEM = "Headers OK";
const char erroropen[] PROGMEM = "Error opening file";
const char error[] PROGMEM = "ERROR ERROR ERROR";
const char initialisesd[] PROGMEM = "Initialising SD card...";
const char dateerror[] PROGMEM = "Dates are not the same - create new file";
const char locate[] PROGMEM = "Locating devices....";
const char parasitic[] PROGMEM = "Parasitic power is:";
const char reference[] PROGMEM = "The ref number is:";
const char noSD[] PROGMEM = "No SD card present";

#define MAX_STRING 80      // Sets the maximum length of string probably could be lower
char stringBuffer[MAX_STRING];  // A buffer to hold the string when pulled from program memory

//****************INITIALISE ROUTINE******************************
void setup()
{
  Serial.begin(115200);    // Set up a serial output for data display and changing parameters
  
  //******Real Time Clock Set - up********
  // A4 and A5 are used as I2C interface.
  // D2 is connected to CLK OUT from RTC. This triggers an interrupt to take data
  // We need to enable pull up resistors
  pinMode(A4, INPUT);           // set pin to input
  digitalWrite(A4, HIGH);       // turn on pullup resistors
  pinMode(A5, INPUT);           // set pin to input
  digitalWrite(A5, HIGH);       // turn on pullup resistors
  pinMode(2,INPUT);    // Set D2 to be an input for the RTC CLK-OUT   
  //initialise the real time clock
  Rtc_Pcf8563 rtc; 
  
  // Read the reference number from the EEROM
  deviceID[0] = char(EEPROM.read(0));
  deviceID[1] = char(EEPROM.read(1));
  
 
  initialiseSD();    // Inisitalise the SD card
  createfilename();  // Create the corrct filename (from date)

  pinMode(LEDred,OUTPUT);    // Set D5 to be an output LED
  pinMode(cardDetect,INPUT);  // D6 is the SD card detect on pin 6.
 
  //Set up digital data lines
  pinMode(7,INPUT);  
  pinMode(8,INPUT);  
  pinMode(9,INPUT);  
  
  // This section configures the RTC to have a 1Hz output.
  // Its a bit strange as first we read the data from the RTC
  // Then we load it back again but including the correct second flag  
  rtc.formatDate(RTCC_DATE_WORLD);
  rtc.formatTime();
  
  year_int = rtc.getYear();
  day_int = rtc.getDay();
  month_int = rtc.getMonth();  
  hour_int = rtc.getHour();
  min_int = rtc.getMinute();
  sec_int = rtc.getSecond(); 
  
  Wire.begin(); // Initiate the Wire library and join the I2C bus as a master
  Wire.beginTransmission(I2C_RTC); // Select RTC
  Wire.write(0);        // Start address
  Wire.write(0);     // Control and status 1
  Wire.write(0);     // Control and status 2
  Wire.write(DecToBcd(sec_int));     // Second
  Wire.write(DecToBcd(min_int));    // Minute
  Wire.write(DecToBcd(hour_int));    // Hour
  Wire.write(DecToBcd(day_int));    // Day
  Wire.write(DecToBcd(2));    // Weekday
  Wire.write(DecToBcd(month_int));     // Month (with century bit = 0)
  Wire.write(DecToBcd(year_int));    // Year
  Wire.write(0b10000000);    // Minute alarm (and alarm disabled)
  Wire.write(0b10000000);    // Hour alarm (and alarm disabled)
  Wire.write(0b10000000);    // Day alarm (and alarm disabled)
  Wire.write(0b10000000);    // Weekday alarm (and alarm disabled)
  Wire.write(0b10000011);     // Output clock frequency enabled (1 Hz) ***THIS IS THE IMPORTANT LINE**
  Wire.write(0);     // Timer (countdown) disabled
  Wire.write(0);     // Timer value
  Wire.endTransmission();
}

void loop()
{

  // If there has been serial data with power information then we can wr
  // Also want to make sure that there is an SD card present
  if(writedataflag==HIGH)
  {
      
    // Each day we want to write a new file.
    // Compare date with previous stored date, every second
    newdate = String(rtc.formatDate(RTCC_DATE_WORLD));  
    if(newdate != date)
    {
       // If date has changed then create a new file
       createfilename();  // Create the corrct filename (from date)
    }
      
    // Here we create a data string to write
    // Subtracting '0' converts from ASCII to real number
    // The headers are: "Reference, Time, Date, V, Voltage, I, Current, P, Power"
    dataString =  String(deviceID[0]); 
    dataString += deviceID[1];
    dataString += comma;
    dataString += newdate;
    dataString += comma;
    dataString += String(rtc.formatTime());
    dataString += comma;
    dataString += dataVoltage;  // This is the data value
    dataString += comma;
    dataString += dataCurrent;  // This is the data value
    dataString += comma;
    dataString += dataPower;  // This is the data value
    
    if(digitalRead(cardDetect)==LOW&&cardDetectOld==HIGH)
    {
      // There was no card previously so re-initialise and re-check the filename
      initialiseSD();
      createfilename();
    }
    
    if(digitalRead(cardDetect)==LOW&&cardDetectOld==LOW)
    {
      //Ensure that there is a card present)
      digitalWrite(LEDred, HIGH);   // set the LED ON
      // We then write the data to the SD card here:
      writetoSD();
      digitalWrite(LEDred, LOW);   // set the LED ON
    }
    else
    {
       Serial.println(getString(noSD));
      // print to the serial port too:
      Serial.println(dataString);
    }
    
    if(digitalRead(cardDetect)==HIGH)
    {
      // This meands there is no card present so set he LED high
       digitalWrite(LEDred, HIGH);   // set the LED ON
    }     

    cardDetectOld = digitalRead(cardDetect);  // Store the old value of the card detect
    writedataflag=LOW;    // Reset the write data flag
  }
  
  //*********Here we check to see if any serial data has been seen***********
  getData();    // Check the serial port for data
  sortData();   // Sort the data for useful information
  delay(50);
  
}

//*********** FUNCTION TO INITIALISE THE SD CARD***************
void initialiseSD()
{
  Serial.println(getString(initialisesd));
  // make sure that the default chip select pin is set to
  // output, even if you don't use it:
  pinMode(chipSelect, OUTPUT);

  // see if the card is present and can be initialized:
  if (!SD.begin(chipSelect)) {
    //Serial.println("FAIL");
    // don't do anything more:
    // Want to turn on an ERROR LED here
    return;
  }
}


// *********FUNCTION TO SORT OUT THE FILENAME**************
void createfilename()
{
  // Check there is a file created with the date in the title
  // If it does then create a new one with the new name
  // The name is created from:
  // RXXDMMDD.CSV, where XX is the reference, MM is the month, DD is the day
  // You must add on the '0' to convert to ASCII
  
  filename[1]=deviceID[0];  
  filename[2]=deviceID[1];   
  date = String(rtc.formatDate());
  day_int = rtc.getDay();  // Get the actual day from the RTC
  month_int = rtc.getMonth();  // Get the month
  day_int1 = day_int/10;    // Find the first part of the integer
  day_int2 = day_int%10;    // Find the second part of the integer
  filename[4]=day_int1 + '0';  // Convert from int to ascii
  filename[5]=day_int2 + '0';  // Convert from int to ascii 
  month_int1 = month_int/10;    // Find the first part of the integer
  month_int2 = month_int%10;    // Find the second part of the integer
  filename[6]=month_int1 + '0';  // Convert from int to ascii
  filename[7]=month_int2 + '0';  // Convert from int to ascii   

  // Create the file and put a row of headers at the top
  if (! SD.exists(filename)) {
    // only open a new file if it doesn't exist
    datafile = SD.open(filename, FILE_WRITE);    // Open the correct file
    // if the file is available, write to it:
    if (datafile) {
      datafile.println(getString(headers));
      datafile.close();
      Serial.println(getString(headersOK));
    }  
    // if the file isn't open, pop up an error:
    else {
      Serial.println(getString(erroropen));
    }  
  }
}


// This routine writes the dataString to the SD card
void writetoSD()
{
  datafile = SD.open(filename, FILE_WRITE);    // Open the correct file
  // if the file is available, write to it:
  if (datafile) {
    datafile.println(dataString);
    datafile.close();
    // print to the serial port too:
    Serial.println(dataString);
  }  
  // if the file isn't open, pop up an error:
  else {
    Serial.println(getString(erroropen));
  }
}

// This routine pulls the string stored in program memory so we can use it
// It is temporaily stored in the stringBuffer
char* getString(const char* str) {
	strcpy_P(stringBuffer, (char*)str);
	return stringBuffer;
}

// Converts a decimal to BCD (binary coded decimal)
byte DecToBcd(byte value){
  return (value / 10 * 16 + value % 10);
}

// **********************GET DATA SUBROUTINE*****************************************
// This sub-routine picks up and serial string sent to the device and sorts out a power string if there is one
// All values are global, hence nothing is sent/returned

void getData()
{
  // **********GET DATA*******************************************
  // We want to find the bit of interesting data in the serial data stream
  // As mentioned above, we are using LLAP for the data.
  // All the data arrives as serial commands via the serial interface.
  // All data is in format aXXDDDDDDDDD where XX is the device ID
  if (Serial.available() > 0) 
  {
    inByte = Serial.read(); // Read whatever is happening on the serial port
  
    if(inByte=='a')    // If we see an 'a' then read in the next 11 chars into a buffer.
    {   
      str_buffer+=inByte;
      for(int i = 0; i<11;i++)  // Read in the next 11 chars - this is the data
      {
        inByte = Serial.read(); 
        str_buffer+=inByte;
      }
      Serial.println(str_buffer);  // TEST - print the str_buffer data (if it has arrived)
      sortData();
      str_buffer="";  // Reset the buffer to be filled again
    }
  }
}

// **********************SORT DATA SUBROUTINE*****************************************
// This sub-routine takes the read-in data string (12 char, starting with a) and does what is required with it
// The str-buffer is global so we do not need to send it to the routine

void sortData()
{ 
  // Here we want to be able to ask the device what its ID is
  // “aXXID?-----“
  // It does not matter what XX are
  if(str_buffer.substring(3,6) == "ID?")
  {
    Serial.print(getString(reference));
    Serial.println(deviceID);
  } 
  
  // For all other commands we first want to check if the device ID matches.
  // If it does not then we disregard the command (as it was not meant for this device      
  if(str_buffer.substring(1,3) == deviceID)
  {
    // If yes then we can do further checks on ths data
    // This is where we do all of the checks on the incomming serial command:
    
    //Serial.println("Matches Device ID");  // TEST - got into this routine
    
    // Change device ID:
    // Device ID
    // “aXXCHIDXXE--“
    // Where the last two values (XX) are the new device ID (from AA to ZZ).
    if(str_buffer.substring(3,7) == "CHID")
    {
      // First check if the NEW device ID is within the allowable range (AA-ZZ)
      // to do this we can convert to an int and check if it is within the OK levels
      // A -> int is 65, Z -. int is 90
      // So our new device ID as an int must be between 65 and 90 for it to be valid
      if(65<=int(str_buffer[7])&&int(str_buffer[7])<=90&&65<=int(str_buffer[8])&&int(str_buffer[8])<=90)   
      { // If is all OK then write the data
        // Change device ID
        Serial.print("Change Device ID to ");
        Serial.println(str_buffer.substring(7,9));  // This will change the device ID
        deviceID[0] = str_buffer[7];
        deviceID[1] = str_buffer[8];
        // Also want to store this into EEPROM
        EEPROM.write(0, deviceID[0]);    // Do this seperately
        EEPROM.write(1, deviceID[1]);
        initialiseSD();
        createfilename();             
      }
      else
      {
        Serial.println("Invalid device ID"); 
     }      
    } 
    
    // Receive Power Data in format “aXXP????-----“ 
    // Where XX is the reference and ???? is the data
    else if(str_buffer.substring(3,4) == "P")
    {
      //Serial.print("Power:");
      //Serial.println(str_buffer.substring(4,8));
      dataPower = str_buffer.substring(4,8);
      writedataflag=HIGH;  // Set the unit to write the data
    } 
 
     // Receive Power Data in format “aXXV???-----“ 
    // Where XX is the reference and ??? is the data
    else if(str_buffer.substring(3,4) == "V")
    {
      //Serial.print("Voltage:");
      //Serial.println(str_buffer.substring(4,7));
      dataVoltage = str_buffer.substring(4,6);
      dataVoltage += '.';      
      dataVoltage += str_buffer.substring(6,7);
      
      //writedataflag=HIGH;  // Set the unit to write the data
    }
        // Receive Current Data in format “aXXI????-----“ 
    // Where XX is the reference and ???? is the data
    else if(str_buffer.substring(3,4) == "I")
    {
      //Serial.print("Current:");
      //Serial.println(str_buffer.substring(4,8));
      dataCurrent = str_buffer.substring(4,8);
      //writedataflag=HIGH;  // Set the unit to write the data
    }
    
  }
  else
  {
    //Serial.println("Device ID does not match");
  }
}

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

371

372

373

374

375

376

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

393

394

395

396

397

398

399

400

401

402

403

404

405

406

407

408

409

410

411

412

413

414

415

416

417

418

419

420

421

422

423

424

425

426

427

428

429

430

431

432

433

434

435

436

437

438

439

440

441

442

443

444

445

446

447

448

449

450

451

452

453

454

455

456

457

458

459

460

461

462

463

464

465

466

467

468

469

470

471

472

473

474

475

476

477

478

479

480

481

482

483

484

485

486

487

488

489

490

491

492

493

494

495

496

497

498

499

500

501

502

503

504

505

506

507

508

509

510

511

512

513

514

515

516

517

518

519

/********************************************************

/****** DataDuino - Arduino Serial DAQ UNIT *************

/****** by Matt Little **********************************

/****** Date: 30/10/12 **********************************

/****** info@re-innovation.co.uk ************************

/****** www.re-innovation.co.uk *************************

/********************************************************

See www.re-innovation.co.uk for information and construction details

This is the SERIAL data storage version.

This is sample code for the DataDuino PCB.

/*************Details of Code*****************************

The DataDuino is an Arduino based SD card datalogger.

A PCF8563 Realt Time Clock is used to timestamp the data.

Data is written every time there is serial data in the correct format.

Data is written to a .csv file created on an SD card.

A new file is created each day. If file alreay present then data is appended.

The file name is created from the reference number and the date in the format:

RXXDXXXX.csv, where RXX is the reference number and DXXXX is the date in the format DDMM.

SERIAL Commands (at 115200 baud) :

"aXXCHID??E--"

This will change the reference number to ??

"a--ID?----"

This will return the device ID

Updates:

30/10/12 Code written - Matt Little

31/10/12 writing code description and sorting out new sample timings - Matt Little

19/9/13 Getting code to work on Arduino 1.0.5 - Matt Little

19/9/13 Updated the Rtc_Pcf8563 to work with Arduino 1.0.5 - Matt Little

19/9/13 Changed getData and sortData to 'listen' for serial data - Matt Little

19/9/13 Removed all references to temperature - not needed - Matt Little

23/9/13 Changed format of data - Matt Little

//*********SD CARD DETAILS***************************

The SD card circuit:

SD card attached to SPI bus as follows:

** MOSI - pin 11

** MISO - pin 12

** CLK - pin 13

** CS - pin 10

** Card detect - pin 6

SD card code details:

created 24 Nov 2010

updated 2 Dec 2010

by Tom Igoe

//************ Real Time Clock code*******************

A PCF8563 RTC is attached to pins:

** A4 - SDA (serial data)

** A5 - SDC (serial clock)

** D2 - Clock out - This gives a 1 second pulse to record the data

RTC PCF8563 code details:

By Joe Robertson, jmr

orbitalair@bellsouth.net

**********************************************************************************************************/

/************ External Libraries*****************************/

#include <stdlib.h>

#include <Wire.h> // Required for RTC

#include <Rtc_Pcf8563.h> // RTC library

#include <SD.h> // SD card library

#include <avr/pgmspace.h> // Library for putting data into program memory

#include <EEPROM.h> // For writing values to the EEPROM

#include <avr/eeprom.h>

/************User variables and hardware allocation**********************************************/

/******* SD CARD*************/

const int chipSelect = 10; // The SD card Chip Select pin 10

const int cardDetect = 6; // The SD card detect is on pin 6

// The other SD card pins (D11,D12,D13) are all set within SD.h

int cardDetectOld = LOW; // This is the flag for the old reading of the card detect

/*************Real Time Clock*******/

Rtc_Pcf8563 rtc;

#define I2C_RTC 0x51 // 7 bit address (without last bit - look at the datasheet)

int RTCinterrupt = 0; // RTC interrupt - This is pin 2 of ardunio - which is INT0

/********* Output LED *************/

const int LEDred = 5; // The output led is on pin 5

/********* Digital Data Storage ***********/

int D7 = 0; // Digital input

int D8 = 0; // Digital input

int D9 = 0; // Digital input

/********** Analogue Data Storage ************/

int analog0 = 0; // Analog input

int analog1 = 0; // Analog input

int analog2 = 0; // Analog input

int analog3 = 0; // Analog input

// ****** Serial Data Read***********

// Variables for the serial data read

char inByte; // incoming serial char

String str_buffer = ""; // This is the holder for the string which we will display

//********Variables for the Filename*******************

char filename[] = "RXXDXXXX.csv"; // This is a holder for the full file name

//int refnumber; // The house number here, which is stored in EEPROM

char deviceID[3]; // A buffer to hold the device ID

File datafile; // The logging file

String dataString; // This is the holder for the data as a string. Start as blank

int counter = 0; // Clue is in the name - its a counter.

long dataCounter = 0; // This holds the number of seconds since the last data store

long sampleTime = 10; // This is the time between samples for the DAQ

// Variables for the Pulse Counter

int pulseinterrupt = 1; // Pulse Counter Interrupt - This is pin 3 of arduino - which is INT1

volatile long pulsecounter = 0; // This counts pulses from the flow sensor - Needs to be long to hold number

volatile long pulsecounterold = 0; // This is storage for the old flow sensor - Needs to be long to hold number

volatile int writedataflag = HIGH; // A flag to tell the code when to write data

// Varibales for writing to EEPROM

int hiByte; // These are used to store longer variables into EERPRPROM

int loByte;

// These next ints are for the filename conversion

int day_int =0; // To find the day from the Date for the filename

int day_int1 =0;

int day_int2 =0;

int month_int = 0;

int month_int1 = 0;

int month_int2 = 0;

int year_int = 0; // Year

int hour_int = 0;

int min_int = 0;

int sec_int = 0;

//**********STRINGS TO USE****************************

String comma = ",";

String date; // The stored date from filename creation

String newdate; // The new date, read every time

//String dataType; // This holds the dataType ("P", "V", "I")

String dataVoltage; // This holds the actual data

String dataCurrent; // This holds the actual data

String dataPower; // This holds the actual data

// These are Char Strings - they are stored in program memory to save space in data memory

// These are a mixutre of error messages and serial printed information

const char headers[] PROGMEM = "Ref,Date,Time,Voltage,Current,Power"; // Headers for the top of the file

const char headersOK[] PROGMEM = "Headers OK";

const char erroropen[] PROGMEM = "Error opening file";

const char error[] PROGMEM = "ERROR ERROR ERROR";

const char initialisesd[] PROGMEM = "Initialising SD card...";

const char dateerror[] PROGMEM = "Dates are not the same - create new file";

const char locate[] PROGMEM = "Locating devices....";

const char parasitic[] PROGMEM = "Parasitic power is:";

const char reference[] PROGMEM = "The ref number is:";

const char noSD[] PROGMEM = "No SD card present";

#define MAX_STRING 80 // Sets the maximum length of string probably could be lower

char stringBuffer[MAX_STRING]; // A buffer to hold the string when pulled from program memory

//****************INITIALISE ROUTINE******************************

void setup()

{

Serial.begin(115200); // Set up a serial output for data display and changing parameters

//******Real Time Clock Set - up********

// A4 and A5 are used as I2C interface.

// D2 is connected to CLK OUT from RTC. This triggers an interrupt to take data

// We need to enable pull up resistors

pinMode(A4, INPUT); // set pin to input

digitalWrite(A4, HIGH); // turn on pullup resistors

pinMode(A5, INPUT); // set pin to input

digitalWrite(A5, HIGH); // turn on pullup resistors

pinMode(2,INPUT); // Set D2 to be an input for the RTC CLK-OUT

//initialise the real time clock

Rtc_Pcf8563 rtc;

// Read the reference number from the EEROM

deviceID[0] = char(EEPROM.read(0));

deviceID[1] = char(EEPROM.read(1));

initialiseSD(); // Inisitalise the SD card

createfilename(); // Create the corrct filename (from date)

pinMode(LEDred,OUTPUT); // Set D5 to be an output LED

pinMode(cardDetect,INPUT); // D6 is the SD card detect on pin 6.

//Set up digital data lines

pinMode(7,INPUT);

pinMode(8,INPUT);

pinMode(9,INPUT);

// This section configures the RTC to have a 1Hz output.

// Its a bit strange as first we read the data from the RTC

// Then we load it back again but including the correct second flag

rtc.formatDate(RTCC_DATE_WORLD);

rtc.formatTime();

year_int = rtc.getYear();

day_int = rtc.getDay();

month_int = rtc.getMonth();

hour_int = rtc.getHour();

min_int = rtc.getMinute();

sec_int = rtc.getSecond();

Wire.begin(); // Initiate the Wire library and join the I2C bus as a master

Wire.beginTransmission(I2C_RTC); // Select RTC

Wire.write(0); // Start address

Wire.write(0); // Control and status 1

Wire.write(0); // Control and status 2

Wire.write(DecToBcd(sec_int)); // Second

Wire.write(DecToBcd(min_int)); // Minute

Wire.write(DecToBcd(hour_int)); // Hour

Wire.write(DecToBcd(day_int)); // Day

Wire.write(DecToBcd(2)); // Weekday

Wire.write(DecToBcd(month_int)); // Month (with century bit = 0)

Wire.write(DecToBcd(year_int)); // Year

Wire.write(0b10000000); // Minute alarm (and alarm disabled)

Wire.write(0b10000000); // Hour alarm (and alarm disabled)

Wire.write(0b10000000); // Day alarm (and alarm disabled)

Wire.write(0b10000000); // Weekday alarm (and alarm disabled)

Wire.write(0b10000011); // Output clock frequency enabled (1 Hz) ***THIS IS THE IMPORTANT LINE**

Wire.write(0); // Timer (countdown) disabled

Wire.write(0); // Timer value

Wire.endTransmission();

}

void loop()

{

// If there has been serial data with power information then we can wr

// Also want to make sure that there is an SD card present

if(writedataflag==HIGH)

{

// Each day we want to write a new file.

// Compare date with previous stored date, every second

newdate = String(rtc.formatDate(RTCC_DATE_WORLD));

if(newdate != date)

{

// If date has changed then create a new file

createfilename(); // Create the corrct filename (from date)

}

// Here we create a data string to write

// Subtracting '0' converts from ASCII to real number

// The headers are: "Reference, Time, Date, V, Voltage, I, Current, P, Power"

dataString = String(deviceID[0]);

dataString += deviceID[1];

dataString += comma;

dataString += newdate;

dataString += comma;

dataString += String(rtc.formatTime());

dataString += comma;

dataString += dataVoltage; // This is the data value

dataString += comma;

dataString += dataCurrent; // This is the data value

dataString += comma;

dataString += dataPower; // This is the data value

if(digitalRead(cardDetect)==LOW&&cardDetectOld==HIGH)

{

// There was no card previously so re-initialise and re-check the filename

initialiseSD();

createfilename();

}

if(digitalRead(cardDetect)==LOW&&cardDetectOld==LOW)

{

//Ensure that there is a card present)

digitalWrite(LEDred, HIGH); // set the LED ON

// We then write the data to the SD card here:

writetoSD();

digitalWrite(LEDred, LOW); // set the LED ON

}

else

{

Serial.println(getString(noSD));

// print to the serial port too:

Serial.println(dataString);

}

if(digitalRead(cardDetect)==HIGH)

{

// This meands there is no card present so set he LED high

digitalWrite(LEDred, HIGH); // set the LED ON

}

cardDetectOld = digitalRead(cardDetect); // Store the old value of the card detect

writedataflag=LOW; // Reset the write data flag

}

//*********Here we check to see if any serial data has been seen***********

getData(); // Check the serial port for data

sortData(); // Sort the data for useful information

delay(50);

}

//*********** FUNCTION TO INITIALISE THE SD CARD***************

void initialiseSD()

{

Serial.println(getString(initialisesd));

// make sure that the default chip select pin is set to

// output, even if you don't use it:

pinMode(chipSelect, OUTPUT);

// see if the card is present and can be initialized:

if (!SD.begin(chipSelect)) {

//Serial.println("FAIL");

// don't do anything more:

// Want to turn on an ERROR LED here

return;

}

// *********FUNCTION TO SORT OUT THE FILENAME**************

void createfilename()

{

// Check there is a file created with the date in the title

// If it does then create a new one with the new name

// The name is created from:

// RXXDMMDD.CSV, where XX is the reference, MM is the month, DD is the day

// You must add on the '0' to convert to ASCII

filename[1]=deviceID[0];

filename[2]=deviceID[1];

date = String(rtc.formatDate());

day_int = rtc.getDay(); // Get the actual day from the RTC

month_int = rtc.getMonth(); // Get the month

day_int1 = day_int/10; // Find the first part of the integer

day_int2 = day_int%10; // Find the second part of the integer

filename[4]=day_int1 + '0'; // Convert from int to ascii

filename[5]=day_int2 + '0'; // Convert from int to ascii

month_int1 = month_int/10; // Find the first part of the integer

month_int2 = month_int%10; // Find the second part of the integer

filename[6]=month_int1 + '0'; // Convert from int to ascii

filename[7]=month_int2 + '0'; // Convert from int to ascii

// Create the file and put a row of headers at the top

if (! SD.exists(filename)) {

// only open a new file if it doesn't exist

datafile = SD.open(filename, FILE_WRITE); // Open the correct file

// if the file is available, write to it:

if (datafile) {

datafile.println(getString(headers));

datafile.close();

Serial.println(getString(headersOK));

}

// if the file isn't open, pop up an error:

else {

Serial.println(getString(erroropen));

}

// This routine writes the dataString to the SD card

void writetoSD()

{

datafile = SD.open(filename, FILE_WRITE); // Open the correct file

// if the file is available, write to it:

if (datafile) {

datafile.println(dataString);

datafile.close();

// print to the serial port too:

Serial.println(dataString);

}

// if the file isn't open, pop up an error:

else {

Serial.println(getString(erroropen));

}

// This routine pulls the string stored in program memory so we can use it

// It is temporaily stored in the stringBuffer

char* getString(const char* str) {

strcpy_P(stringBuffer, (char*)str);

return stringBuffer;

}

// Converts a decimal to BCD (binary coded decimal)

byte DecToBcd(byte value){

return (value / 10 * 16 + value % 10);

}

// **********************GET DATA SUBROUTINE*****************************************

// This sub-routine picks up and serial string sent to the device and sorts out a power string if there is one

// All values are global, hence nothing is sent/returned

void getData()

{

// **********GET DATA*******************************************

// We want to find the bit of interesting data in the serial data stream

// As mentioned above, we are using LLAP for the data.

// All the data arrives as serial commands via the serial interface.

// All data is in format aXXDDDDDDDDD where XX is the device ID

if (Serial.available() > 0)

{

inByte = Serial.read(); // Read whatever is happening on the serial port

if(inByte=='a') // If we see an 'a' then read in the next 11 chars into a buffer.

{

str_buffer+=inByte;

for(int i = 0; i<11;i++) // Read in the next 11 chars - this is the data

{

inByte = Serial.read();

str_buffer+=inByte;

}

Serial.println(str_buffer); // TEST - print the str_buffer data (if it has arrived)

sortData();

str_buffer=""; // Reset the buffer to be filled again

}

// **********************SORT DATA SUBROUTINE*****************************************

// This sub-routine takes the read-in data string (12 char, starting with a) and does what is required with it

// The str-buffer is global so we do not need to send it to the routine

void sortData()

{

// Here we want to be able to ask the device what its ID is

// “aXXID?-----“

// It does not matter what XX are

if(str_buffer.substring(3,6) == "ID?")

{

Serial.print(getString(reference));

Serial.println(deviceID);

}

// For all other commands we first want to check if the device ID matches.

// If it does not then we disregard the command (as it was not meant for this device

if(str_buffer.substring(1,3) == deviceID)

{

// If yes then we can do further checks on ths data

// This is where we do all of the checks on the incomming serial command:

//Serial.println("Matches Device ID"); // TEST - got into this routine

// Change device ID:

// Device ID

// “aXXCHIDXXE--“

// Where the last two values (XX) are the new device ID (from AA to ZZ).

if(str_buffer.substring(3,7) == "CHID")

{

// First check if the NEW device ID is within the allowable range (AA-ZZ)

// to do this we can convert to an int and check if it is within the OK levels

// A -> int is 65, Z -. int is 90

// So our new device ID as an int must be between 65 and 90 for it to be valid

if(65<=int(str_buffer[7])&&int(str_buffer[7])<=90&&65<=int(str_buffer[8])&&int(str_buffer[8])<=90)

{ // If is all OK then write the data

// Change device ID

Serial.print("Change Device ID to ");

Serial.println(str_buffer.substring(7,9)); // This will change the device ID

deviceID[0] = str_buffer[7];

deviceID[1] = str_buffer[8];

// Also want to store this into EEPROM

EEPROM.write(0, deviceID[0]); // Do this seperately

EEPROM.write(1, deviceID[1]);

initialiseSD();

createfilename();

}

else

{

Serial.println("Invalid device ID");

}

// Receive Power Data in format “aXXP????-----“

// Where XX is the reference and ???? is the data

else if(str_buffer.substring(3,4) == "P")

{

//Serial.print("Power:");

//Serial.println(str_buffer.substring(4,8));

dataPower = str_buffer.substring(4,8);

writedataflag=HIGH; // Set the unit to write the data

}

// Receive Power Data in format “aXXV???-----“

// Where XX is the reference and ??? is the data

else if(str_buffer.substring(3,4) == "V")

{

//Serial.print("Voltage:");

//Serial.println(str_buffer.substring(4,7));

dataVoltage = str_buffer.substring(4,6);

dataVoltage += '.';

dataVoltage += str_buffer.substring(6,7);

//writedataflag=HIGH; // Set the unit to write the data

}

// Receive Current Data in format “aXXI????-----“

// Where XX is the reference and ???? is the data

else if(str_buffer.substring(3,4) == "I")

{

//Serial.print("Current:");

//Serial.println(str_buffer.substring(4,8));

dataCurrent = str_buffer.substring(4,8);

//writedataflag=HIGH; // Set the unit to write the data

}

else

{

//Serial.println("Device ID does not match");

}

One response to “DataDuino – Serial”

Hanul Chung says:

3rd January 2018 at 21:29

Hello,

Would it be possible for us to get one unit of your I-V tracer?
I have a 300 watt CPV module to test.
Please let me know the cost and lead-time if available.

Talk to you soon.

Reply

Concept

SD write cycles

Code flow overview

Arduino code for the Dataduino

One response to “DataDuino – Serial”

Leave a Reply Cancel reply