SD Card Example?

Toit

#SD Card Example?

Thread channel in help

Oliver M 02/26/2023 07:16 PM

Hi, I'm trying to store some data on an SD Card. I have found the flash library, which provides Mount, which allows me to mount (and format, if necessary) an SD Card. But how do I read and write with it? Are there any examples that I could use as a starting off point?

floitsch 02/26/2023 09:35 PM

The host package has file and directory operations.

floitsch 02/26/2023 09:36 PM

I haven't tested it myself, but @MikkelD implemented it, and the host package's functions should map to sdcard accesses.
We still need to update the package description...

MikkelD 02/27/2023 06:08 AM

We discussed renaming the host package filesystem. Maybe it should even be considered to move it into the standard library, seeing that we keep getting this question?

Oliver M 02/27/2023 08:47 AM

So I just mount the SD card, and then treat it just like a file system on my Linux machine?

MikkelD 02/27/2023 09:21 AM

yes

theHuanter 02/27/2023 01:53 PM

I used it like this:

sdcard = SDCard 
    --miso=gpio.Pin 19
    --mosi=gpio.Pin 23
    --clk=gpio.Pin 18
    --cs=gpio.Pin 5

sdcard.openw filename
sdcard.write "something"
sdcard.close

using this abstraction file:

import flash
import gpio 
import spi
import host.file

class SDCard:

  csvfile/file.Stream := ?

  constructor --miso/gpio.Pin --mosi/gpio.Pin --clk/gpio.Pin --cs/gpio.Pin --mount_point/string="/sd":
    bus := spi.Bus
      --miso=miso
      --mosi=mosi
      --clock=clk

    sdcard := flash.Mount.sdcard
      --mount_point=mount_point
      --spi_bus=bus
      --cs=cs

    csvfile = file.Stream.for_write "/sd/test"
    csvfile.close
    print "SDCard initialised"

  write line/string filename/string:
    print "write \"$line\" into file: $filename"
    csvfile = file.Stream.for_write filename
    csvfile.write line
    csvfile.close

  openw filename:
    csvfile = file.Stream.for_write filename

  openr filename:
    csvfile = file.Stream.for_read filename

  write line/string:
    csvfile.write line

  read_file filename -> string:
    return (file.read_content filename).to_string

  close:
    csvfile.close

theHuanter

I used it like this: ``` sdcard = SDCard --miso=gpio.Pin 19 --mosi=gpio.Pin 23 --clk=gpio.Pin 18 --cs=gpio.Pin 5 sdcard.openw filename sdcard.write "something" s...

Oliver M 02/27/2023 02:09 PM

Thanks, will try that out!

Oliver M 03/12/2023 06:20 PM

OK, I've now tried this, but I'm getting an error:

E (175187) sdmmc_sd: sdmmc_init_sd_if_cond: send_if_cond (1) returned 0x108
E (175187) vfs_fat_sdmmc: sdmmc_card_init failed (0x108).

******************************************************************************
Decoding by `jag`, device has version <2.0.0-alpha.55>
******************************************************************************
EXCEPTION error. 
UNKNOWN ERROR 0x108(264)
  0: init_sdcard_              <sdk>/flash.toit:141:3
  1: Mount.sdcard              <sdk>/flash.toit:44:14
  2: SDCard                    sdtest.toit:18:15
  3: main                      sdtest.toit:55:13
******************************************************************************

Oliver M 03/12/2023 06:21 PM

Pins: MISO-32, MOSI-23, CS-25, CLOCK-33

Oliver M 03/12/2023 06:21 PM

Any idea what is going wrong? Am I using a pin I shouldn't be using?

theHuanter 03/12/2023 06:32 PM

unknown error is not very helpfull tbh

what esp are you using and how did you connected the sd reader?

Oliver M 03/12/2023 06:34 PM

I'm using a TTGO T-BEAM, and have just wired it to the respective pins.

theHuanter 03/12/2023 06:36 PM

is the sdcard formatted and working?

Oliver M 03/12/2023 06:41 PM

It should be formatted on initialisation.

Oliver M 03/12/2023 07:03 PM

Ah, if I switch the MOSI and MISO pins, I'm getting a different error:

E (22087) sdmmc_sd: sdmmc_check_scr: send_scr returned 0x107
E (22087) vfs_fat_sdmmc: sdmmc_card_init failed (0x107).

******************************************************************************
Decoding by `jag`, device has version <2.0.0-alpha.55>
******************************************************************************
EXCEPTION error. 
UNKNOWN ERROR 0x107(263)
  0: init_sdcard_              <sdk>/flash.toit:141:3
  1: Mount.sdcard              <sdk>/flash.toit:44:14
  2: SDCard                    sdtest.toit:18:15
  3: main                      sdtest.toit:56:13
******************************************************************************

Oliver M 03/12/2023 07:17 PM

Right, now I'm using MISO/4, MOSI/13, CLOCK/14, and CS/2 (which I found someone else on the web use), and the first error is gone; now I only get

E (58017) vfs_fat_sdmmc: sdmmc_card_init failed (0x107).

******************************************************************************
Decoding by `jag`, device has version <2.0.0-alpha.55>
******************************************************************************
EXCEPTION error. 
UNKNOWN ERROR 0x107(263)
  0: init_sdcard_              <sdk>/flash.toit:141:3

(edited)

Oliver M 03/12/2023 07:34 PM

(I'm using an 8GB SD card, which works fine on my Linux machine)

Oliver M 03/12/2023 07:38 PM

And it's formatted as FAT32

theHuanter 03/12/2023 07:53 PM

MikkelD 03/13/2023 07:56 AM

#define ESP_ERR_TIMEOUT 0x107 /*!< Operation timed out */

I think that is what you see. That would suggest that the SPI protocol receives a timeout. I would double check the pins and their connections.

Oliver M 03/13/2023 11:10 AM

I’m pretty sure they are fine, as I switched pins several times. I’ve got another reader module, so will try with that one.

theHuanter 03/13/2023 02:03 PM

For SPI you have this two options with the pins. In your case the MOSI should be 13 instead of 4. I think only the CS pin can be whatever you wish but would also make it 15 for testing.

theHuanter 03/13/2023 02:05 PM

it doesn't really matter which board you are using - as long as it is an ESP32 the pins should be the same. Of course you can also use software SPI with defining any pin but I am not sure how good this will work out

theHuanter 03/13/2023 02:05 PM

VSPI is the default

theHuanter 03/13/2023 02:06 PM

you could also test it using arduino code (they have example code for SDCards) just to make sure SPI works.

Oliver M 03/13/2023 02:39 PM

My problem is that on the board I’m using (T-BEAM), not all those pins are exposed, so I have no choice but use other ones

Oliver M 03/13/2023 03:20 PM

Is there any example of doing software SPI in toit?

floitsch 03/13/2023 03:20 PM

What do you mean with "software SPI" ?
Edit. Ah. I see InformaticOre's response now.(edited)

theHuanter

it doesn't really matter which board you are using - as long as it is an ESP32 the pins should be the same. Of course you can also use software SPI with defining any pin but I am n...

floitsch 03/13/2023 03:24 PM

The ESP32 can use pretty much any pin for the SPI. It might not be able to run at full speed when it has to reroute the pins, but that's almost never a problem.

floitsch 03/13/2023 03:26 PM

Here is a simplified version of the SPI allocation:

// Check if there is a preferred device.
  if ((mosi == -1 || mosi == 13) &&
      (miso == -1 || miso == 12) &&
      (clock == -1 || clock == 14)) {
    host_device = HSPI_HOST;
  }
  if ((mosi == -1 || mosi == 23) &&
      (miso == -1 || miso == 19) &&
      (clock == -1 || clock == 18)) {
    host_device = VSPI_HOST;
  }

So if you use these pins, you would get the optimal assignment, but it's really not necessary.

floitsch

The ESP32 can use pretty much any pin for the SPI. It might not be able to run at full speed when it has to reroute the pins, but that's almost never a problem.

theHuanter 03/13/2023 03:26 PM

thats what I meant

afaik using the dedicated pins it is doing hardware SPI and using any other Pins will do a software SPI. Some devices might have issues with that but I also don't know any.

floitsch 03/13/2023 03:26 PM

It's not a software SPI.

floitsch 03/13/2023 03:26 PM

It's just running the pin through a matrix.

floitsch 03/13/2023 03:27 PM

There should be no CPU overhead doing that.

Oliver M 03/13/2023 03:27 PM

So can I use arbitrary pins for mosi/miso/clock? I'm confused now. I can't use either the HSPI or VSPI pins.

floitsch 03/13/2023 03:28 PM

From the docs.

Most of ESP32’s peripheral signals have direct connection to their dedicated IO_MUX pins. However, the signals can also be routed to any other available pins using the less direct GPIO matrix. If at least one signal is routed through the GPIO matrix, then all signals will be routed through it.

The GPIO matrix introduces flexibility of routing but also brings the following disadvantages:

- Increases the input delay of the MISO signal, which makes MISO setup time violations more likely. If SPI needs to operate at high speeds, use dedicated IO_MUX pins.

- Allows signals with clock frequencies only up to 40 MHz, as opposed to 80 MHz if IO_MUX pins are used.

Oliver M 03/13/2023 03:29 PM

Ah, would that be why I'm getting a time-out on the SD card?

floitsch 03/13/2023 03:29 PM

I doubt it.

theHuanter 03/13/2023 03:29 PM

is there a default clock speed in toit? can you set it to 40k?

floitsch 03/13/2023 03:31 PM

I think you can set the frequency when getting a device.

theHuanter 03/13/2023 03:32 PM

yea, from the toit docs:

import gpio
import spi

main:
  bus := spi.Bus
    --miso=gpio.Pin 12
    --mosi=gpio.Pin 13
    --clock=gpio.Pin 14

  device := bus.device
    --cs=gpio.Pin 15
    --frequency=10_000_000

floitsch 03/13/2023 03:33 PM

Try with a really low frequency, like 1MHz.

floitsch 03/13/2023 03:33 PM

That should eliminate hardware issues.

floitsch 03/13/2023 03:38 PM

Also: do you use any adapter for the SD card?

Oliver M 03/13/2023 03:46 PM

I've got this one: https://www.amazon.co.uk/dp/B077MCQS9P?psc=1&ref=ppx_yo2ov_dt_b_product_details

AZDelivery Micro SD SPI Storage Board TF Memory Card Adapter Shield...

AZDelivery Micro SD SPI Storage Board TF Memory Card Adapter Shield Module 3.3V 5V compatible with Arduino Including E-Book! (Pack of 3)

Oliver M 03/13/2023 03:46 PM

(I'm using it on 5V, as I have read that 3.3V can lead to problems)

floitsch 03/13/2023 03:51 PM

yes. that should work.

floitsch 03/13/2023 03:52 PM

I would start by trying to run it a lower speed.

floitsch 03/13/2023 03:55 PM

I think there is no default speed. You have to specify the frequency.

floitsch 03/13/2023 03:56 PM

Which pins are you currently using?

Oliver M 03/13/2023 04:04 PM

I’ve tried a variety. Will have another go this evening with lower speed. It’s only going to be a data logger, so speed is not important anyway.

Oliver M 03/13/2023 09:00 PM

How do I pass a frequency to the Mount? Do I create a Device with the frequency (which I then ignore)? SD Card is the only option which hasn't got a frequency parameter.

Oliver M 03/13/2023 09:05 PM

(If I read the docs right, it's the device config)

OPOliver M

How do I pass a frequency to the Mount? Do I create a Device with the frequency (which I then ignore)? SD Card is the only option which hasn't got a frequency parameter.

floitsch 03/13/2023 09:07 PM

You are right. The sdcard part doesn't have any frequency.

@MikkelD

is there maybe a way to test the sdcard with the other flash functions? Like opening it as nand and reading from it? Wouldn't need to understand the filesystem for that.

Oliver M 03/13/2023 09:14 PM

Now getting error 0x102. Not long until I've got them all through

Oliver M 03/13/2023 09:14 PM

This was with defining a bus.device with a frequency of 1MHz

Oliver M 03/13/2023 09:17 PM

With 2MHz it's 0x109.

floitsch 03/13/2023 09:17 PM

That's an "invalid argument" error. (102)(edited)

Oliver M 03/13/2023 09:17 PM

Is there a list of error codes somewhere that I can check?

Oliver M 03/13/2023 09:17 PM

(I did have a look in the toit-master directory)

floitsch 03/13/2023 09:18 PM

https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/error-codes.html

Oliver M 03/13/2023 09:18 PM

So 109 (invalid CRC) might be speed related...

Oliver M 03/14/2023 07:16 PM

I'm trying a different card reader module; I have taken off all other peripherals, and now I consistently get error ESP_ERR_INVALID_CRC If I take the card out of the reader, then I'm getting a timeout (0x107)

Oliver M 03/14/2023 07:18 PM

The same happens whether I try Mount.sdcard or Mount.sdcard_unformatted

Oliver M 03/14/2023 07:22 PM

Using some other pins (mosi=23, miso=32, clock=33, cs=25) I'm now getting 0x108 (invalid response). I give up.(edited)

MikkelD 03/14/2023 08:00 PM

What i think is happening here is that the sd-card has been formatted with something that is not recognised completely by the ESP-IDF, but almost. So it will try to mount it (instead of formatting). We only format if it is blank(ish). At the moment there is no way to force a format. The sdcard readers are normally so dumb that they should not pose an issue. Trying a completely blank sdcard might be worth a try.(edited)

Oliver M 03/14/2023 08:08 PM

It’s one I newly bought, but it was formatted already (in FAT32)…

Oliver M 03/14/2023 08:54 PM

No, I reformatted it, and even created a smaller (2GB) partition instead of the single 8GB one. Still no success.

theHuanter 03/14/2023 10:30 PM

strange for me it worked pretty wel. it was even woorrking better then any other SDCard implementation of arduino or micrropython where I had also a loott of troubles. The ony difference is, I am using the SDCard directy without any level shifting or adapter. just plain SPI.

You could try and skip the module and solder some cables to a SD Card adapter, the small things for micro SD Cards which look like normal big SDCards.
Or do you have a plain ESP32 DEV Board? where you can use the dedicated SPI pins? just for the sake of testing because my board is using the exact pins.
If I remember correctly I also had issues with not using the dedicated SPI pins for SPI but it is already some time ago.

Oliver M 03/15/2023 12:49 PM

Are you saying I could use an adapter instead of the card module?!

floitsch 03/15/2023 12:58 PM

I think @theHuanter means that you can interface the sdcard directly to the pins: https://alexlubbock.com/micro-sd-adapter-esp8266-esp32

Connect a microSD card to the ESP8266 or ESP32 using a microSD to S...

Rather than buy a dedicated breakout, it's straightforward to solder up a microSD connector

floitsch 03/15/2023 12:59 PM

Most of the time I have seen a pull-up on MISO, though.

floitsch 03/15/2023 01:00 PM

Like here: https://www.mischianti.org/2021/03/28/how-to-use-sd-card-with-esp32-2/

floitsch 03/15/2023 01:00 PM

Fundamentally, your board is doing exactly the same.

floitsch 03/15/2023 01:01 PM

It also takes the 5V and regulates it down to 3.3V, but that's clearly not necessary if you already have a 3.3V supply.

floitsch 03/15/2023 01:01 PM

not sure if the board also does level shifting.

floitsch 03/15/2023 01:04 PM

Looks like it has a LVC125 buffer, though.

floitsch 03/15/2023 01:04 PM

That separates the sdcard from the microcontroller.

floitsch 03/15/2023 01:07 PM

I wouldn't be surprised if that chip acts as a level shifter.

Oliver M 03/15/2023 01:15 PM

Yes, looking at the SD pins, they can be used in SPI mode. I could always try that, though the adapter board doesn’t really do anything else.

Oliver M 03/15/2023 01:16 PM

The one thing I haven’t tried is using 3.3V instead of 5V; so I will try that this evening.

Oliver M 03/15/2023 01:16 PM

And then maybe solder a spare adapter straight to the ESP

floitsch 03/15/2023 01:21 PM

good luck

floitsch

good luck

Oliver M 03/15/2023 01:24 PM

Tack!

floitsch

Like here: https://www.mischianti.org/2021/03/28/how-to-use-sd-card-with-esp32-2/

Oliver M 03/19/2023 12:55 PM

Some positive news: I have taken an SD card adapter, and soldered wires straight onto it, instead of using the adapter with a break-out board. And 3.3V instead of the 5V the board used (which also doesn't work woith 3.3V either). And, it works. The first error I got was a "File not found", as I had mistyped a pathname. But now it works. And I can even read the file when connecting the SD card to my Linux machine. Happy days.

86 messages in total