SoftwareSerial 7-E-1 problem

I'm trying to connect a smart meter to an arduino to monitor my electricity usage. I'm
using the SoftwareSerial library but the serial port of this meter uses 7 bit, even parity &
1 stop bit with inverse logic and the library uses 8N1.

On the forum I have read solutions (from pylon) with 8E2 and 8O1 but I'm not able to work out
the modifications in the code.

Could someone please help me ?

Jeroen

The changes for 7E1 are:

void SoftwareSerial::recv()
{

#if GCC_VERSION < 40302
// Work-around for avr-gcc 4.3.0 OSX version bug
// Preserve the registers that the compiler misses
// (courtesy of Arduino forum user *etracer*)
  asm volatile(
    "push r18 \n\t"
    "push r19 \n\t"
    "push r20 \n\t"
    "push r21 \n\t"
    "push r22 \n\t"
    "push r23 \n\t"
    "push r26 \n\t"
    "push r27 \n\t"
    ::);
#endif  

  uint8_t d = 0;

  // If RX line is high, then we don't see any start bit
  // so interrupt is probably not for us
  if (_inverse_logic ? rx_pin_read() : !rx_pin_read())
  {
    // Wait approximately 1/2 of a bit width to "center" the sample
    tunedDelay(_rx_delay_centering);
    DebugPulse(_DEBUG_PIN2, 1);

    // Read each of the 7 bits
    for (uint8_t i=0x1; i != 0x80; i <<= 1)
    {
      tunedDelay(_rx_delay_intrabit);
      DebugPulse(_DEBUG_PIN2, 1);
      uint8_t noti = ~i;
      if (rx_pin_read())
        d |= i;
      else // else clause added to ensure function timing is ~balanced
        d &= noti;
    }
    // skip the parity bit
    tunedDelay(_rx_delay_stopbit);
    DebugPulse(_DEBUG_PIN2, 1);

    // skip the stop bits
    tunedDelay(_rx_delay_stopbit);
    DebugPulse(_DEBUG_PIN2, 1);

    if (_inverse_logic)
      d = ~d & 0x7F;

    // if buffer full, set the overflow flag and return
    if ((_receive_buffer_tail + 1) % _SS_MAX_RX_BUFF != _receive_buffer_head) 
    {
      // save new data in buffer: tail points to where byte goes
      _receive_buffer[_receive_buffer_tail] = d; // save new byte
      _receive_buffer_tail = (_receive_buffer_tail + 1) % _SS_MAX_RX_BUFF;
    } 
    else 
    {
#if _DEBUG // for scope: pulse pin as overflow indictator
      DebugPulse(_DEBUG_PIN1, 1);
#endif
      _buffer_overflow = true;
    }
  }

#if GCC_VERSION < 40302
// Work-around for avr-gcc 4.3.0 OSX version bug
// Restore the registers that the compiler misses
  asm volatile(
    "pop r27 \n\t"
    "pop r26 \n\t"
    "pop r23 \n\t"
    "pop r22 \n\t"
    "pop r21 \n\t"
    "pop r20 \n\t"
    "pop r19 \n\t"
    "pop r18 \n\t"
    ::);
#endif
}

and

size_t SoftwareSerial::write(uint8_t b)
{
  if (_tx_delay == 0) {
    setWriteError();
    return 0;
  }

  uint8_t oldSREG = SREG;
  cli();  // turn off interrupts for a clean txmit

  // Write the start bit
  tx_pin_write(_inverse_logic ? HIGH : LOW);
  tunedDelay(_tx_delay + XMIT_START_ADJUSTMENT);
  uint8_t p = 0;
  uint8_t t;
  for (t = 0x40; t; t >>= 1)
    if (b & t) p++;
  

  // Write each of the 7 bits
  if (_inverse_logic)
  {
    for (byte mask = 0x01; mask != 0x80; mask <<= 1)
    {
      if (b & mask) // choose bit
        tx_pin_write(LOW); // send 1
      else
        tx_pin_write(HIGH); // send 0
    
      tunedDelay(_tx_delay);
    }
    // parity
    if (p & 0x01)
      tx_pin_write(HIGH); // send 0
    else 
      tx_pin_write(LOW); // send 1
    tunedDelay(_tx_delay);

    tx_pin_write(LOW); // restore pin to natural state
  }
  else
  {
    for (byte mask = 0x01; mask != 0x80; mask <<= 1)
    {
      if (b & mask) // choose bit
        tx_pin_write(HIGH); // send 1
      else
        tx_pin_write(LOW); // send 0
    
      tunedDelay(_tx_delay);
    }
    // parity
    if (p & 0x01)
      tx_pin_write(LOW); // send 0
    else 
      tx_pin_write(HIGH); // send 1
    tunedDelay(_tx_delay);

    tx_pin_write(HIGH); // restore pin to natural state
  }

  SREG = oldSREG; // turn interrupts back on
  tunedDelay(_tx_delay);
  
  return 1;
}

pylon, thanks for the code !!!

I have made the changes to the SoftwareSerial.cpp and tried to run the following:

#include <SoftwareSerial.h>

SoftwareSerial mySerial(10, 11, true); // RX, TX

void setup()
{
// Open serial communications and wait for port to open:
Serial.begin(9600);
Serial.println("P1 Smart meter reading - START\n");

// set the data rate for the SoftwareSerial port
mySerial.begin(9600);

pinMode(4, OUTPUT);
digitalWrite(4, HIGH);
}

void loop() // run over and over
{
if (mySerial.available())
Serial.write(mySerial.read());
}

But it still returns non-readable information (for me) :

P1 Smart meter reading - START

m?~{og;|{?y88;OO8?8?=??y?y<~c~y~c;8;8;?;8;|;8;y;;;y;{}Oy?8?y?|?y<88>?|yy{8}??<8O8|88?y88??8y{888y?8?;<O8?y>y<m?~{og;|{?y88;OO8?8?=>?y?y<~c~y~c;8;8;?;8;|;8;y;;;y;{}Oy?8?y?|?y<88O8|8~=|y??8}y{{8O8?88O8O88?8O8?8O???O8?8m?~{og;|{?y88;OO8?8?=??y?y<~c~y~c;8;8;?;8;|;8;y;;;y;{}Oy?8?y?|?y<888|8~=|y{y8}??<8O8?88O8??8?88O8?8O8?y??m?~{og;|{?y88;OO8?8?=?>y?y<~c~y~c;8;8;?;8;|;8;y;;;y;{}Oy?8>y?|>y<88O8|8~8|y{y8}??<8O8?888??8?8<O8?8O8?yO8?m?~{og;|{?y88;OO8?8?=??y?y<~c~y~c;8;8;?;8;|;8;y;;;y;{}Oy?8?y?|?y<88??}????888|y{{8}??~8}??<8~O8?<O8?8=8}???y8?;}???Om?~{og;|{?y88;OO8?8?=??y?y<~c~y~c;8;8;?;8;|;8;y;;;y;{}Oy?8?y?|?y<88??O8|y|??8|y{{8}??>8O8O8O8?<O??y}???O8??m?~{og;|{?y88;OO8?8?=??y?y<~c~y~c;8;8;?;8;|;8;y;;;y;{}Oy?8?y?|?y<88??O8|<~{|??8}y{{8}???8O8O88?<O??y}???O8??m?~{og;|{?y88;OO8?8?=??y?y<~c~y~c;8;8;?;8;|;8;y;;;y;{}Oy?8?y?|?y<88??}??O8|888|??8|8=~8}??<88}??<}8=8}8=?y8=;}??;}m?~{og;|{?y88;OO8?8?=??y?y<~c~y~c;8;8;?;8;|;8;y;;;y;{}Oy?8?y?|?y<88??O8??|888|y{?8}??>8O??<O8?<O8=8}??y}??;}???Om?~{og;|{?y88;OO8?8?=>?y?y<~c~y~c;8;8;?;8;|;8;y;;;y;{}Oy?8?y?|?y<88??}??{{O8|<8?|8|8=~8}yy888}?{88?8?888?8O8??O8?m?~{og;|z?y88;OO8?8?=??y?y<~c~y~c;8;8;?;8;|;8;y;;;y;{}Oy?8?y?|?y<88??O8|8~~|??8}y{{8}???8O8O8O8?<O??8}???O8??m?~zog;|{?y88;OO8?8?=??y?y<~c~y~c;8;8;?;8;|;8;y;;;y;{}Oy?8?y?|?y<88??O8|8~~|??8}y{{8O8?88O88888}??888}8=8}8=?y8=;<8=;}O

The output should look like (example) :

/KMP5 ABCD000123456789

0-0:96.1.1(2041424344303030313233343536373839)
1-0:1.8.1(00080.000kWh)
1-0:1.8.2(00039.000
kWh)
1-0:2.8.1(00000.000kWh)
1-0:2.8.2(00000.000
kWh)
0-0:96.14.0(0002)
1-0:1.7.0(0000.22kW)
1-0:2.7.0(0000.00
kW)
0-0:17.0.0(999*A)
0-0:96.3.10(1)
0-0:96.13.1()
0-0:96.13.0()
0-1:24.1.0(3)
0-1:96.1.0(3031323334353637383930313233343536)
0-1:24.3.0(120403080000)(000008)(60)(1)(0-1:24.2.1)(m3)
(00053.271)
0-1:24.4.0(1)
!

Do you have any suggestions ?

Do you have a datasheet for the device you're trying to read? Perhaps we have to adapt other things like activating the pullup on the RX line or inverted start/stop bit logic.

I do not have it specific for the meter I have but it should comply to the following standard :

There is a protocol description on page 10:

Data readout
The Metering System transmits the data message immediately following the activation
through the Request signal. A series of blocks containing the following are sent:

/XXXZ Ident CR LF CR LF Data ! CR LF

Have you tried to use a non-inverted 8N1 serial interface to read the values? From the information you provided and that is available I would also assume that it's 7E1 but you can read that with 8N1 and AND every byte you get with 0x7F, that gets equal results for the reading side.
As for the inversion: I guess you're referring to the definition below the table 4-1 in the linked document where SPACE is defined as >4V and MARK as < 1V. No word is said though about the levels of the start and stop bits. This could mean that you have to read serially without inversion and just XOR the value with 0xFF (NOT operation) to get the bit values inverted but leaving the start and stop bits on their usual level. I hope you understand what I mean.

I'm trying.....

Do you want me to try to use the hardware serial RX or the SoftWareSerial on 8N1 ?
I have tried both but nothing readable.

I didn't expect it to be readable, nevertheless can you post the results?

Interesting would be the results of the hardware serial 8N1 as well as the results of the SoftwareSerial (8N1 too) with inversion activated (that way we have both common types of start/stop bit signaling).

May be a silly question, but do you have a scope to attach to the line to see the signal pattern?

I will do both test again tomorrow and post the results.

Here is some extra information about how other people connected this
meter to their pc with a serial TTL to USB cable (with FTDI chip) :
http://www.domoticaforum.eu/viewtopic.php?f=71&t=6104&start=60

I want to do the same but then with an arduino (mega & ethernet shield).

It's not a silly question but I don't have a scope.

Thank you for your patience, I appreciate your help !!!!

Here are my results, tried to capture three cycles (30sec).

8N1 hardware RX (used example SerialEvent, http://www.arduino.cc/en/Tutorial/SerialEvent)

(eV

eÔRÕd?Zòö?VÖå½Vû??£´ò»ö?'Id?f?ö?V?ö?ö2???f'fff$f'f?¥Öi¶´dë^??$$´Ùò¶I?¼+9½Rû?´dÛ^???ädk??«

¡¥Öi¶?´dë^?????£???«
¡¥Öi¶
?´dÛ^?????£???«
¡¥Ö?¥???£É
û¯????+9½Rû?´2û¯????£???!Ñ)֝i¶?´2û¯????£??«

+9½Vû?$´6û¯?£??«
+9
½Vû??£?£öëZÊÖ?¥???£f´òk)Ö?¥???£f´ök)Ö½å=(eV

eÔRÕd?Zòö?VÖå½Vû??£´ò»ö?'Id?f?ö?V?ö?ö2???f'fff$f'f?¥Öi¶´dë^??$$´Ùò¶I?¼+9½Rû?´dÛ^???ädk???!ñh)֝i¶?´dë^?????£???«

¡¥Öi¶?´dÛ^?????£???«
¡¥Ö?¥???£Éû¯????+9½Rû?´2
û¯????£?f?!Ñ)֝i¶?´2û¯????£??«
+9½Vû?$´6û¯?
£??«
+9½Vó¶?£?£öëZÊÖ?¥???£f´òk)Ö?¥???£f´

8N1 inverted SoftwareSerial (original cpp)

P1 Smart meter reading - START

ÿéûû½þúïå»øú¿ù¸¸»Íϸ½¸º¹¾¾ù¾ù¸üâüùüã»ÿ»¸»¸»½»¸»ü»¸»ÿ»ù»»»ÿ»ù»úùÍù½¸ºù¾ø¾ù¸¸¸ñÿÿ¿½û¸úûÿüùÏù½¸¿ù¿ø¾ú¸¸¸ûø¹¾»þÿÍù¸ú¾¼¸¸¸¸ûý½úøò¸¸¾¸úüϸº¿ù¾¸¸ý¸ùÿ¸¸¸¾ÿý½úÿ¸¸¸¸ûù½ù¾¾¸½¿¸ÿϸ¸¾»ù¸Ï¸¸¾ù¿¸¿¿¾»¸ù±ÿéûÿ½þúíå»üú¿ù¸¸»Ïϸ½¸¾¹¿¾ù¾ù¼ôãüùüã»ÿ»¸»¸»¿»¸»ø»¸»ÿ»ù»»»ÿ»ù»úùÍù½¸ºù¾ü¾ù¸¸¸ùÿÿ¿½ú¸úÿÿüùÍù¿¸¿ù¾ø¾û¸¸¸úø¹¾»úøÏù¾¾¾¸¸¸¸úÿý¸úøú¸¸¾¸ÿüͽ¿¿ù¸¸¸ùù¸ù¾¼¸¾¸ûùù¸ú¾¼¸¸¸ÿù¸ù¾¸¸½¾úÿÏ°¾¾³ùùϸº¿ù¾ý½½¿»¸Ï±ÿéûû½üúíå»øú½ù¸¸»Í͸½¸º¹¾¾ù¾ù¸üâüùüã»ÿ»¸»¸»½»¸»ø»¸»ÿ»ù»»»ÿ³ù»ûùÏù½¸¾ù¿ø¾ù¸¸¸ùÿÿ¾½ú¸ÿûÿøùÍù½¸ºù¾ø¾ú¸¸¸úø½¾úøÿý¸ûüù¸¸¾¸ÿø¿¿¾ú¸¸¸¸þÿ͸¸¹ù¾¸¸ý½ºÿ¸¸¸¿ÿÿ½ºÿ¸¸¸¸úÿù͸°ù¸¸¸¹¸ÿÿ¿¿¾¾¸ùÍ¿¿¾»ùý¸¹¾¿¼ÍÍ

8N1 SoftwareSerial (original cpp)

P1 Smart meter reading - START

(
eÔVGdR
òöV#¯gïVä¶
l2
òd²]¹ÉdÉûöVöööÖÖ¶Òg¯Rä¶ 2äd­òäöÖ2¶ò¶!â¡¥F¯Rä¶ 2ädmòäö¶òlm2v²!â¡¥F¯Rä¶$2äd­òäöööö6äöö¶!â¡¥F¯Rä¶$2ädmòäöööö6äöö¶!ïVä¶ lr2äöäöö¶Rä¶ 22ööö6¶¥¯Rä¶$22öäööö¶ïVä¶ 26öÖö¶!¢ïVä¶ 6 òö ÒVä¶ 6 ò ¥Vä¶ 6äö ¥+Ö( eÔVGdR òöV#¯gïVä¶ l2 òd²]¹ÉdÉûöVöööÖÖ¶Òg¯Rä¶
2äd­òäöÖ2¶ò¶!â¡¥F¯Rä¶
2ädmòäö¶òlm2V¶â¡¥F¯Rä¶$2äd­òäöööö6äöö¶!â¡¥F¯Rä¶$2ädmòäöööö6äöö¶¡ïVä¶
lr2äöäööÒRä¶
22ööö6ö¥¯Rä¶$22öööö¶ïVä¶
2ä6öÖö¶!¢ïVä¶
6
òö
ÒVä¶
6
ò
ïVä¶
6äö
+Ö(
eÔVGdR
òöV#¯gïVä¶
ll2
òd²]`¹ÉdÉûöVöööÖÖ¶Òg¯Rä¶
2äd­òäöÖ2¶ò¶â¡¥F¯Rä¶
2ädmòäö¶ò,m2²!â¡¥F¯Rä¶$2äd­òäöööö6äöö¶!â¡¥F¯Rä¶$2ädmòäöööö6äöö¶¡ïVä¶
lr2äöäö¶Rä¶
22öööÖ$¥¯Rä¶$22äöööö¶ïVä¶

In the thread you linked to, they tried with an Arduino too and were successful in using the hardware serial with a pre-connected inverter. Another one wrote he was successful (at least mostly) with the SoftwareSerial class and changed timings. You might try both but I guess the second options is a lot of trial and error without a scope to get the exact timings.

Do you think it's wise for me to continue with the hardware serial with a pre-connected inverter ?

Yes, I would try that one. The SoftwareSerial does work in some cases but in my experience it produces more problems than it solves. Most people new to Arduino think they can just replace a missing hardware serial with the software version but this usually doesn't work. The hardware version does a lot of stuff not possible in the software version because it would need more processing power than the ATmega series does offer (just one example: timing adaptions based on the received bits).
The SoftwareSerial does work OK for me up 38400 baud if connected to the PC (debugging) but with sensors and other devices I had success only with 1 in 7 cases yet. And even that one case required additional software checks to eliminate the occasional failures.

I understand and will proceed with the hardware serial. Thanks for the lessons learned !!!!

Hi, sorry to bring this thread back, but I have a question (at the end) that is relevant.

I'm using softserial to talk inverted 7E1, 1200 baud (SDI-12 protocol). I used the code changes from this thread

and got the error:
/var/folders/pk/zzpvz5bs0d589468n2bxsyz00000gn/T//ccPjw17q.s: Assembler messages:
/var/folders/pk/zzpvz5bs0d589468n2bxsyz00000gn/T//ccPjw17q.s:243: Error: register r24, r26, r28 or r30 required

Similar to the first post here.

I've applied the fix recommended here:

of changing
: "+r" (delay), "+a" (tmp)

to
: "+w" (delay), "+a" (tmp)

in the tunedDelay function.

My question is why did I need to make this fix in my copy of SoftwareSerial? The regular SoftwareSerial packaged in arduino 1.0.1 does not have this change and compiles fine. diff-ing the code, I made no changes to SoftwareSerial beyond the recv() and write() functions to convert to 7E1.

I had another fix for the same problem by declaring two more variable (parameters for the assembler routine) volatile. This has to do with compiler internals. The compiler keeps care that processor registers are used where appropriate but variables are stored in memory if a register is not suitable. It's feasible that the register allocation changes if there are code changes. If everything is coded correctly this doesn't change anything but it's possible that the declarations in the code are ambiguous and the code compiles anyway. If you then change only a small part of the code you may end in a non-compilable program.
The problem here arises from the combination of machine code (assembler) with C++ code.

JBee:
[...]
Serial.begin(9600);
Serial.println("P1 Smart meter reading - START\n");
[...]
Do you have any suggestions ?

The IEC62056-21 standard dictates an initial rate of 300 baud [edit: sorry, should have looked better, your're trying an electrical interface, I made an optical one, 300 baud refers to the optical standard within IEC62056]. One can negiotiate a higher baud rate after initial communication but that is not needed for readout only.

Some info for interfacing with smart meters:
Type of transmission:
Asynchronous serial bit (Start – Stop) transmission according to ISO/IEC 1177:1985, half-duplex.
Transmission speed:
Initial baud rate – 300
Standard baud rates – 300, 600, 1 200, 2 400, 4 800, 9 600, 19 200.
Character format:
Character format according to ISO/IEC 1177:1985.
(1 start bit, 7 data bits, 1 parity bit, 1 stop bit).
Character code:
Character code according to ISO/IEC 646:1991, international reference version. For local use,
a national replacement code can be used.

How about this?

char ch = Serial.read();
ch = ch >> 1;

SurferTim:
How about this?

char ch = Serial.read();

ch = ch >> 1;

Serial data usually is transfered LSB first, so the parity bit will end in the 8th bit:

char c = Serial.read();
ch &= 0x7F;

Thanks, pylon. You are correct. If it came big endian, and the receiver was expecting little endian, it would take more than just a bit shift. The bit order would need to be reversed, then your code.