Small SD card code?

ah!

I for one would think a boot from SD feature would be good, it would be a great way to do field upgrades.


Rob

of course,
if you format the card on your PC, then upload the boot image file
it SHOULD always start at the same address on the card
all you need to work out is the length
that ought to simplify the problem quite a bit

and if you are in control of the boot image format (and why not!)
you could even store the size at the front of the file

Have you tried chaN fatfs? FatFs - Generic FAT Filesystem Module

How much flash could be allowed for SD read?

I think raw SD read could be quit small if specialized for the avr processor.

If you limited the file to being in the root directory and contiguous, a lot of code would be eliminated.

Files are contiguous unless you have deleted other files and the SD is nearly full.

I will look at what can be done.

Edit: I did some tests and for a 2GB standard SD card formatted FAT16 It takes about 550 bytes of flash to initialize the card, read the master boot record, volume boot block, and first directory block.

Here is the raw init/read code for a 328. It is not very neat, just a prototype test.

#include <avr/io.h>
/** SEND OPERATING CONDITIONS */
uint8_t const ACMD41   = 0X29;
uint8_t const CMD0 = 0;
//uint8_t const CMD8 = 8;
/** READ_BLOCK */
uint8_t const CMD17    = 0X11;
/** APP_CMD - escape for application specific command */
uint8_t const CMD55    = 0X37;

uint8_t const R1_IDLE_STATE = 1;
uint8_t const R1_READY_STATE = 0;
// start data token for read or write
uint8_t const DATA_START_BLOCK = 0XFE;

uint8_t const SS_BIT = 2;
uint8_t const MOSI_BIT = 3;
uint8_t const MISO_BIT = 4;
uint8_t const SCK_BIT = 5;
uint8_t SdCommand(uint8_t cmd, uint32_t arg);
//------------------------------------------------------------------------------
// inline SPI functions
/** Send a byte to the card */
static void spiSend(uint8_t b) {SPDR = b; while(!(SPSR & (1 << SPIF)));}
/** Receive a byte from the card */
static uint8_t spiRec(void) {spiSend(0XFF); return SPDR;}
/** Set Slave Select high */
static void chipSelectHigh(void) {
  PORTB |= (1 << SS_BIT);
}
/** Set chip select low */
static void chipSelectLow(void) {
  PORTB &= ~(1 << SS_BIT);
}
//------------------------------------------------------------------------------
static bool waitForToken(uint8_t token) {
  for (uint8_t i = 0; i != 0XFF; i++) {
    if (spiRec() == token) return true;
  }
  return false;
}
//------------------------------------------------------------------------------
static uint8_t SdAcmd(uint8_t cmd, uint32_t arg) {
  SdCommand(CMD55, 0);
  return SdCommand(cmd, arg);
}
//------------------------------------------------------------------------------
uint8_t SdCommand(uint8_t cmd, uint32_t arg) {
  uint8_t r1;
  // select card
  chipSelectLow();
  
  waitForToken(0XFF);
  // send command
  spiSend(cmd | 0x40);

  // send argument
  for (int8_t s = 24; s >= 0; s -= 8) spiSend(arg >> s);

  // send CRC
  uint8_t crc = 0XFF;
  if (cmd == CMD0) crc = 0X95; // correct crc for CMD0 with arg 0
//  if (cmd == CMD8) crc = 0X87; // correct crc for CMD8 with arg 0X1AA
  spiSend(crc);

  // wait for response
  for (uint8_t retry = 0; ((r1 = spiRec()) & 0X80) && retry != 0XFF; retry++);

  return r1;
}
//------------------------------------------------------------------------------
bool SdInit() {
  uint8_t r;
  uint16_t retry = 0;
  DDRB = (1 << SS_BIT) | (1 << MOSI_BIT) | (1 << SCK_BIT);
  chipSelectHigh();
  // Enable SPI, Master, clock rate f_osc/64
  SPCR = (1 << SPE) | (1 << MSTR) | (1 << SPR1);
  
 // must supply min of 74 clock cycles with CS high.
  for (uint8_t i = 0; i < 10; i++) spiSend(0XFF);
  
  // command to go idle in SPI mode
  while ((r = SdCommand(CMD0, 0)) != R1_IDLE_STATE) {
    if (++retry == 0) return false;
  }
  // start initialization and wait for completed initialization
  while ((r = SdAcmd(ACMD41, 0)) != R1_READY_STATE) {
    if (++retry == 0) return false;
  }
  chipSelectHigh();
  return true;
}
//------------------------------------------------------------------------------
bool SdRead(uint32_t blockNumber, uint8_t* dst) {
  if (SdCommand(CMD17, blockNumber << 9)) {
    return false;
  }
  // wait for start of data
  if (!waitForToken(DATA_START_BLOCK)) return false;
  for (uint16_t i = 0; i < 512; i++) {
    dst[i] = spiRec();
  }
  // first CRC byte
  spiRec();
  // second CRC byte
  spiRec();  
  chipSelectHigh();
  return true;
}

Here is the include to make it a library:

#include <avr/io.h>
bool SdInit();
bool SdRead(uint32_t blockNumber, uint8_t* dst);

And here is a test sketch to read the first directory block:

#include <SdBoot.h>
uint8_t block[512];

#ifdef DBG_PRINT
void printBlock() {
  for( uint16_t i = 0; i < 512; i++) {

    if ((i & 15) == 0){
      Serial.println();
    }else {
      Serial.write(' ');
    }
    Serial.print(block[i] >> 4, HEX);
    Serial.print(block[i] & 15, HEX);    
  }
  Serial.println();
  
}
#endif  // DBG_PRINT
//------------------------------------------------------
void setup() {
  bool b;
#ifdef DBG_PRINT
  Serial.begin(9600);
  while (!Serial.available());
#endif  // DBG_PRINT
  b = SdInit();
//  Serial.println(b, DEC);
  // read MBR
  b = SdRead(0, block);
//  Serial.println(b, DEC);
  // address of FAT volume
  uint32_t s = *(uint32_t*)&block[454];
  // read volume boot block
  b = SdRead(s, block);
  // FAT size
  uint16_t f = *(uint16_t*)&block[22];
  // address of directory
  uint32_t d = s + 2*f + 1;
  // read first block of directory
  b = SdRead(d, block);
#ifdef DBG_PRINT
  printBlock();
#endif  // DBG_PRINT
}
void loop() {}

Here is a more complete SD read program that might be small enough to use in a boot loader. You need to login to download the attached file.

It works with FAT16 formatted standard SD cards on 168/328 Arduinos. It opens a file and reads it a byte at a time.

It appears to take about 1200 bytes of flash. This sketch takes 1618 bytes on 0022.

#include <SdBoot.h>
void setup() {
  // open TEST.BIN
  fileOpen((uint8_t*)"TEST    BIN");
  // read a byte
  fileRead();
}
void loop() {}

An empty sketch takes 450 bytes:

void setup() {}
void loop() {}

Here is the readme.txt file:

Here is a small file read program. It appears to take about 1200 bytes of flash.

It could be optimized more and the retry timeouts need adjusting.

It runs on a 168/328 Arduino with an SD module that uses pin 10 for chip select.

To change the chip select pin edit this section of SdBoot.cpp

// use pin 10 for chip select
#define CS_DDR DDRB
#define CS_PORT PORTB
uint8_t const CS_BIT = 2;

It is packaged as a library for testing.

Just copy the SdBoot folder to your libraries folder and run the examples.

There are three examples:

writeTest.pde write a four MB test file. It requires SdFat.

readTest.pde reads the test file.

sizeTest.pde shows approximately how much flash is used.

SdBoot20120201.zip (5.62 KB)

It appears to take about 1200 bytes of flash.

Thanks! I think 1200 bytes is definitely small enough to be interesting. That would mean (probably) a 2k bootloader (same as the original arduino bootloader) that could do both serial and SD.

Um. Was there supposed to be a link in that message?

That is awesome! Combine 512 byte standard bootloader with 1200 byte bootload from SD card if CD signal is detected on a '1284 would be a great combo.

westfw,

The library is an attachment to the message with the readme.txt quote. You must be logged in to see it.

@fat16lib
just trying it with IDE 1.0
just a couple of bytes bigger

word of advice for a dummy, please:
my datalogger shield has CS on pin D4

what should my settings be:
#define CS_DDR DDRB
#define CS_PORT PORTB
uint8_t const CS_BIT = 2;

your "tiny" sketch takes 1634 bytes with IDE 1.0, so only 16 bytes bigger!

I have tested the sketches with an Enet shield which has pin 4 as SD chip select. The sketches are designed to disable the Enet SPI part which uses pin 10 as chip select.

This code currently works only on 168/328 Arduinos.

In writeTest change init to this:

  if (!sd.init(4)) {

To run readTest edit SdBoot.cpp and change the chip select defs like this:

// use pin 4 for chip select
#define CS_DDR DDRD
#define CS_PORT PORTD
uint8_t const CS_BIT = 4;

Remember this is not well tested code. I am trying to help westfw and hope he will pursue an SD boot loader.

File names are always eleven bytes, alpha are caps.

The first eight bytes contain the file name with blank fill.

The last three bytes contain the file extension with blank fill.

Thanks I'll try it first thing in the morning
Cheers
Mike

as interesting as the idea is. wouldn't it be easier to take the time to write out a 'full' (i'm using this term loosely) bootloader similar to a PC's embeded chip that allows it to boot the ram/hdd?

it seems like a bootloader could be written that could not only boot the sd but also there should be enough room left over to throw in an interpreter for c files...

i just figure that if someone can write a functional 3 line (30 mb) program that can check the answers to calculus equations. then it should be able to be done. and before anyone says that doing this would leave no room on the chip...i want to say "YOUR RIGHT!!! IT WOULDN'T" but considering you would then be able to use a sd card..which i beleive do come in gb....you wouldn't need the eprom anymore...

in a way doing this would also cause the arduino to be classified as a true micro computer...and not just a chipset or...yah...forgot the others...

anyways...my point is...if i got a basic outline of such a bootloader...would anyone be willing to aide in perfecting it?

All three Sdboot examples worked well in Arduino Uno (328P) with an Ethernet shield. A 2 gigabyte Sd -card of a not a well known manufacturer. Arduino 1.0. Windows XP. The only thing was to change the pin nr in the library as instructed earlier.
No hardware errors reported either.
sizeTest program file (flash) size was 1634 bytes .
File TEST.BIN was read in 210 seconds with readTest.

This code is something I had hoped to be written, and it happened sooner than I expected. Excellent work.

Thanks to fat16lib!

A new version of SdBoot is attached to this reply. You must be logged in to download the the attachment.

I fixed a couple of bugs. The most serious bug required a major change to avoid a 32-bit multiply and the increase in flash size.

I added a number of configuration options and improved error handling. I managed to limit the size increase to six bytes by optimizing several areas.

The options such as chip select pin can be set by editing SdBootConfig.h.

SdBoot20120202.zip (8.43 KB)

The new version worked with the configuration file. sizeTest 1644 bytes. readTest time in fast mode 85 seconds.

In future we might transfer new versions of application programs via internet or phone to an SD card file of Arduino, anywhere in the world, and Arduino would update (= replace) its own application program in the program memory using the boot program.

All
This sounds very interesting but I fail to feel confident about what this is about. So I have some questions.

I think this is about a sketch that allows to load and run a sketch written on a sd card. The total size of the Arduino bootloader and the sketch from fat16lib and the sketch on the sd card can not be bigger than the total amount of flash memory on your Arduino.

Assuming the above is correct. What I don't understand is:
Is the sketch from fat16lib some code I include in my sketch or is it a sketch on its own.
When the sketch from fat16lib has uploaded the SD card content and I remove the sd card and reset arduino what is running on the arduino? the sdcard code or the sketch from fat16lib?
The code on the sdcard is this a hex file? If not how do I create it?

Best regards
Jantje

The total size of the Arduino bootloader and the sketch from fat16lib and the sketch on the sd card can not be bigger than the total amount of flash memory on your Arduino.

Correct, except that the program written by fat16lib (or something similar) should be added to the bootloader so the Arduino can boot from an SD card. So there would only be two programs, the BL and the application sketch.

Is the sketch from fat16lib some code I include in my sketch or is it a sketch on its own.

The program written by fatlib16 is a file reader, if added to a bootloader the lot will reside in the upper portion of the flash memory. In general a bootloader will get a file from somewhere (UART or SD or outer space, it doesn't matter) and burn the contents of that file into lower flash memory.

Having done so it will jump to the newly burned program.

Now I think we started talking about the bootloader testing for a file on the SD at power up, if it finds the file it does the above. If not it simply jumps into the resident program.

What optimistx is talking about it taking it a step further. On a given trigger the bootloader could download a file from say the internet, burn it into the SD card then restart the chip. This would cause the above mechanism to be invoked and voila you can update your gadget from 10000k away with no need to mail an SD card or have the user burn one themselves.

Is the sketch from fat16lib some code I include in my sketch or is it a sketch on its own.

He has provided some utility functions that can be used in your programs, plus a few test sketches.

When the sketch from fat16lib has uploaded the SD card content and I remove the sd card and reset arduino what is running on the arduino? the sdcard code or the sketch from fat16lib?

As I understand it the code is just a file reader not a bootloader. So it doesn't write to the flash at all, so if you load one of the test sketches that's what will always run regardless of how many times you read a file from the SD.

The code on the sdcard is this a hex file?

At present I think it simply reads any file, it could be the first chapter of "War and Peace". When/if this is incorporated into a bootloader the file format can be HEX or binary or anything, however HEX would be the most likely.

I've not been following this for a few days and just caught up, apologies if I have the wrong end of the stick.


Rob

Rob
Thanks for the explanation.
It is lots clearer now. I'm still not 100% understanding but I guess that is because we are in the early stages.
Best regards
Jan