I2S Audio on ESP32-WROVER-E

Toit

#I2S Audio on ESP32-WROVER-E

Thread channel in help

Nathan 02/18/2023 02:58 AM

Hello! I have an ESP32-LyraT v4.3 development board which has a ESP32-WROVER-E and ES8388 codec chip on-board. I am trying to get audio from the AUX_IN to the PHONEJACK. Using the I2S library, I'm trying to do basic reads/writes to/from an I2S bus but I guess I am missing something, because I only seem to be able to read 0x00 out of the bus.

Here's what I have so far:

import i2s
import gpio

SCLK ::= gpio.Pin 5
LRCK ::= gpio.Pin 25
DSDIN ::= gpio.Pin 26
ASDOUT ::= gpio.Pin 35

main:
    // Initialize I2S bus

    bus := i2s.Bus --sck=SCLK --ws=LRCK --tx=DSDIN --rx=ASDOUT --sample_rate=44100 --bits_per_sample=16 --buffer_size=32
    i := 255

    while i-- > 0:
        bytes := bus.read
        print "Writing $(bytes.size) bytes"
        print "$bytes"
        bus.write bytes

Any advice would be much appreciated!

floitsch 02/18/2023 03:01 AM

@MikkelD

is using Toit with audio. He might have some idea.
In the meantime let me check out the specs for the ES8388

Nathan 02/18/2023 03:02 AM

Appreciate it!

Nathan 02/18/2023 03:04 AM

https://espressif-docs.readthedocs-hosted.com/projects/esp-adf/en/latest/design-guide/dev-boards/get-started-esp32-lyrat.html

has the documentation for the board and links to the ES8388 pdf as well

floitsch 02/18/2023 03:14 AM

From what I can see from the ES8388 datasheet you still need to configure the chip over i2c (or SPI).
There are 53 registers that can be configured. Starting the conversions might be as simple as setting one register to the correct value, or it could be complicated.
I'm going to search a bit more, but it's already pretty late here, so I doubt I will make much progress.

floitsch 02/18/2023 03:20 AM

I found this user guide for the ES8388. (edit: forgot to add the link. https://dl.radxa.com/rock2/docs/hw/ds/ES8388%20user%20Guide.pdf )
Without too much confidence, I would say that the ADC is powered down by default, and would need to be enabled by setting the corresponding bits in register 3. (Page 16).

You can see some typical startup sequences on pages 20+.(edited)

floitsch 02/18/2023 03:22 AM

If you look at page 26, section 10.3, it looks like setting the registers shouldn't be too hard.
I'm not really familiar with all of the terminology, but there are only a few terms that would need some googling. Most of the registers could just be set directly according to the flow of that page.

floitsch 02/18/2023 03:24 AM

If you want to try to set the registers, have a look at this tutorial that explains how to write a driver: https://docs.toit.io/peripherals/drivers/sparkfun_joystick

Nathan 02/18/2023 03:34 AM

thanks for the pointers, I am fairly green when it comes to programming hardware but I will take a look

Nathan 02/18/2023 03:35 AM

the ESP-ADF does a lot of behind the scenes things as part of their audio-passthru example

https://github.com/espressif/esp-adf/blob/master/examples/audio_processing/pipeline_passthru/main/passthru.c

but I couldn't tell how much of it was C boilerplate that is abstracted away by some of toit niceties

floitsch 02/18/2023 03:49 AM

It looks like the es8388 is accessed using i2c on pins 18 and 23: https://github.com/espressif/esp-adf/blob/master/components/audio_board/lyrat_v4_3/board_pins_config.c#L38

esp-adf/board_pins_config.c at master · espressif/esp-adf

Espressif Audio Development Framework. Contribute to espressif/esp-adf development by creating an account on GitHub.

floitsch 02/18/2023 03:51 AM

And here seems to be the start routine for the chip: https://github.com/espressif/esp-adf/blob/master/components/audio_hal/driver/es8388/es8388.c#L154

floitsch 02/18/2023 03:53 AM

Probably need the init configuration first: https://github.com/espressif/esp-adf/blob/master/components/audio_hal/driver/es8388/es8388.c#L263

Nathan 02/18/2023 03:54 AM

yes I guess I will go down the rabbit hole and try to get a basic driver going, here is what I have

import gpio
import i2c

SCL ::= gpio.Pin 23
SDA ::= gpio.Pin 18

ES8388_ADDR ::= 0x20

main:
    bus := i2c.Bus --scl=SCL --sda=SDA

    device := bus.device ES8388_ADDR

floitsch 02/18/2023 03:54 AM

Looks good.
Make sure to do a scan on the bus to see if you got the id correct.

Nathan 02/18/2023 03:54 AM

also this pins config is very helpful because I couldn't find it documented elsewhere...

floitsch 02/18/2023 03:54 AM

And that the chip responds

floitsch 02/18/2023 03:57 AM

It's bed time for me now, but I will check tomorrow again. So just keep posting and I (and probably Mikkel) will respond tomorrow.

Nathan 02/18/2023 03:58 AM

thanks again, will see how far I can get

Nathan 02/18/2023 05:38 AM

I'm not sure why but my i2c.Bus scan is returning 16 (0x10) when according to the documentation, the ES8388 should be on 0x20

Nathan 02/18/2023 06:01 AM

looks like the documentation is wrong.. if i try to read from 0x20 I get I2C read errors, but on 0x10 I seem to get register defaults that resemble that of an ES8388

MikkelD 02/18/2023 09:51 AM

So, my two cents. You are on the right track. The ES8388 needs to be configured over i2c to enable the ADC. Also, it needs to be setup with the same I2S parameters as your toit I2S. One thing to consider, now that you are looking, would be to release the ES8388 toit configuration as a toit driver? @floitsch can help you with that.

Nathan 02/18/2023 10:28 AM

sure if I manage to get it working I would be more than happy to share it, however I am not sure what the bar is in terms of quality

OPNathan

looks like the documentation is wrong.. if i try to read from 0x20 I get I2C read errors, but on 0x10 I seem to get register defaults that resemble that of an ES8388

floitsch 02/18/2023 11:04 AM

That's a common problem with i2c. When reading from i2c one takes the ID and shifts it. When writing one shifts and adds one.
The datasheets often just say that the ID is the shifted value. Libraries use the unshifted value.

Nathan 02/18/2023 11:05 AM

heh, well good to know

floitsch 02/18/2023 11:06 AM

Happens often ...
One of the reasons I recommended the scan.

Nathan 02/18/2023 11:06 AM

the ESP-ADF driver has some "mystery meat" in it..

https://github.com/espressif/esp-adf/blob/master/components/audio_hal/driver/es8388/es8388.c#L277-L280

I am not sure where these registers come from, I have a mind to not include them in my attempt at a toit driver port

OPNathan

sure if I manage to get it working I would be more than happy to share it, however I am not sure what the bar is in terms of quality

floitsch 02/18/2023 11:07 AM

Sharing is always good. Whether it's worth making it a registered package is a different question and can be done later.

OPNathan

the ESP-ADF driver has some "mystery meat" in it.. https://github.com/espressif/esp-adf/blob/master/components/audio_hal/driver/es8388/es8388.c#L277-L280 I am not sure where the...

floitsch 02/18/2023 11:10 AM

Personally I try to go with datasheet and application notes first, and then fall back to other implementations when something isn't working as expected.
It depends a bit in on what you want to get out of it. By starting with datasheet and application notes you tend to get a better understanding. With the code you often are faster in getting it to work.

Nathan 02/18/2023 11:11 AM

Yes I have been going back and forth between the two, these registers don't appear in the datasheet.. If I can get sound in/out of the DAC/ADC I'll be pretty happy, then I figure I can always add more things later

floitsch 02/18/2023 11:12 AM

Exactly

floitsch 02/18/2023 11:13 AM

And congratulations. You seem to be doing quite well for something who is green in programming hardware.

Nathan 02/18/2023 11:17 AM

Slow progress and I'm sure to get stuck quite soon

floitsch 02/18/2023 11:17 AM

floitsch 02/18/2023 11:19 AM

Just curious: which timezone are you in?

Nathan 02/18/2023 11:33 AM

I'm in Australia

floitsch 02/18/2023 11:35 AM

Explains the times you are online.

Nathan 02/18/2023 11:35 AM

yes

Nathan 02/18/2023 01:08 PM

Unfortunately I'm somewhat stuck again.. back to getting zeroes out of the i2s bus... here is what my driver looks like:

https://github.com/c22/toit-scratch/blob/main/ES8388_driver.toit

and here is me trying to read out of the ADC using the driver:

https://github.com/c22/toit-scratch/blob/main/basic-audio.toit

floitsch 02/18/2023 01:10 PM

looking.

floitsch

I found this user guide for the ES8388. (edit: forgot to add the link. https://dl.radxa.com/rock2/docs/hw/ds/ES8388%20user%20Guide.pdf ) Without too much confidence, I would say th...(edited)

floitsch 02/18/2023 01:14 PM

Seems like I didn't attach the user-guide I found...
https://dl.radxa.com/rock2/docs/hw/ds/ES8388%20user%20Guide.pdf

Nathan 02/18/2023 01:14 PM

floitsch 02/18/2023 01:20 PM

Just going from the user guide, I would look into setting register 0 (REG_CHIP_CONTROL_1).

Nathan 02/18/2023 01:20 PM

looks like at least one thing I missed is putting in slave mode

floitsch 02/18/2023 01:20 PM

I think it's in slave mode by default.

floitsch 02/18/2023 01:21 PM

and tbh, this was one of the areas I would need to google a bit, to know exactly what it means and how it is set up on the board.

floitsch 02/18/2023 01:21 PM

(possibly taking some hints from the existing code).

floitsch 02/18/2023 01:25 PM

A lost resort is also to modify the arduino code and add printfs around the i2c calls to see which calls are made. It's a bit annoying to switch IDEs, and setups, so I tend to avoid it. Not sure if I should. It would probably speed up debugging these things.

Nathan 02/18/2023 01:27 PM

hmm I will try twiddling with some stuff in register 0 then might need to pack it in for the day and try again later.. thanks for the pointers!

floitsch 02/18/2023 01:28 PM

This repository seems to much more minimal than the code from espressif.
https://github.com/thaaraak/es8388

GitHub - thaaraak/es8388

Contribute to thaaraak/es8388 development by creating an account on GitHub.

Nathan 02/18/2023 01:29 PM

one problem is half the things in the datasheet I don't actually understand

For example "ADC Fs is the same as DAC Fs" option is set by ESP-ADF code, but I don't know what "Fs" are...

floitsch 02/18/2023 01:29 PM

welcome to my world

floitsch 02/18/2023 01:30 PM

fwiw, it's a rewarding experience.
After writing drivers I often feel like I'm a super expert. (Probably shouldn't but let me have that

floitsch 02/18/2023 01:31 PM

Fiddling with NFC boards (mfrc-522 and similar) right now, and the things I learned...

Nathan 02/18/2023 01:31 PM

hehe, I just wanted to use something nicer than the ESP-IDF.. somehow writing drivers in a language I've never even used before seems more inviting than that...

floitsch 02/18/2023 01:32 PM

lol

floitsch 02/18/2023 01:32 PM

I will order an es8388 now, but since I usually order from aliexpress, it will take many weeks before it will arrive.

Nathan 02/18/2023 01:34 PM

an interesting aspect of jaguar is that, in theory, one could make their device available to anyone in the world for building/testing.. though at the moment I am relying heavily on being able to read the COM port outputs but I imagine that is because I'm not using everything toit/jaguar has to offer

floitsch 02/18/2023 01:35 PM

In v1 the print output was automatically sent to the server and then the command-line.

floitsch 02/18/2023 01:35 PM

I did actually fiddle with a sensor that was located in South Africa

floitsch 02/18/2023 01:36 PM

The system is set up in a way that prints don't need to go through the serial port, and we have plans to change Jaguar so it uses that feature.(edited)

floitsch 02/18/2023 01:37 PM

If the device was publicly accessible (public IP), Jaguar could then actually do what you suggested.

floitsch 02/18/2023 01:37 PM

(jag scan <ip> avoids the udp broadcast scanning and sets the target to the given IP)

Nathan 02/18/2023 01:37 PM

yes I could port forward it

Nathan 02/18/2023 01:38 PM

I imagine it's not intended for that just yet, but it's doable

floitsch 02/18/2023 01:38 PM

Exactly.

floitsch 02/18/2023 01:38 PM

And tbh, I wouldn't have the time to look into it right now anyway. But it's a cool feature that already came in handy multiple times (on v1).(edited)

Nathan 02/18/2023 01:39 PM

yes the ability to also collab with a friend and give them access to run code on my device while we tinker from our own homes is really nice

MikkelD 02/19/2023 10:59 AM

@floitsch

i have a spare lyra board if you want it. Fs is the sample rate. Since the lyra board is kind of the espressif official audio board it will make sense to have drivers for it publicly.

floitsch 02/19/2023 11:03 AM

Thanks. I already ordered one, and should arrive relatively soon.
If it turns out to be a blocker for @Nathan , having a look a bit earlier might make sense though. I can't promise I will find the time, but definitely nothing will happen until I actually have a physical board ...
In summary: if you are in the neighborhood anyway, maybe drop one off. But don't do a trip just for it.

MikkelD 02/19/2023 01:05 PM

Right, so register 8 bit 7 needs to be zero, to set the I2S in slave mode. In codec lingo "serial interface" refers to I2S.(edited)

Nathan 02/19/2023 11:01 PM

I've added

// Set slave serial port mode
reg = registers_.read_u8 REG_DAC_POWER_MANAGEMENT
reg &= ~0b1000_0000
registers_.write_u8 REG_DAC_POWER_MANAGEMENT reg

I also added

// Enable internal power up/down sequence
reg = registers_.read_u8 REG_CHIP_CONTROL_1
reg |= 0b0000_1000
registers_.write_u8 REG_CHIP_CONTROL_1 reg

but it didn't seem to help, and the reference drivers don't mess with this bit so i reset and removed it for now

Nathan 02/19/2023 11:03 PM

still only getting zeroes, I think I am at the point where I probably need to compare register by register to the reference driver and copy everything over until it works, even if the values don't really make sense to me

floitsch 02/19/2023 11:03 PM

Annoying

Nathan 02/19/2023 11:06 PM

also I'm not an experienced bit twiddler so entirely possible I have OR/AND mixed up somewhere..

it is about this time where I would typically start logging EVERYTHING...

I have a question about printing int values but I'll ask in general

Nathan 02/20/2023 12:03 AM

I just realised when adding logging that I was setting slave mode on the wrong register

floitsch 02/20/2023 12:03 AM

we have all been there...

Nathan 02/20/2023 12:04 AM

Unfortunately setting it on the correct register did not help, but I noticed something strange.. When I try to use my driver from my "test" code, none of the print statements are printing.. Maybe I am initializing the driver wrong somehow?

floitsch 02/20/2023 12:05 AM

The print should not be different.

floitsch 02/20/2023 12:05 AM

That's definitely weird.

Nathan 02/20/2023 12:07 AM

if I run the driver directly (driver has a main function) then I get print statements, if I include the driver and run

codec := ES8388 device
    codec.on

then I don't get the print statements from the driver, so I am clearly instantiating the driver incorrectly

floitsch 02/20/2023 12:07 AM

hmm.

floitsch 02/20/2023 12:07 AM

I would start by adding a print into the constructor and into on.

floitsch 02/20/2023 12:08 AM

is your repository up to date?

Nathan 02/20/2023 12:12 AM

no about to update it, but I must've been just not reading it right.. or it mysteriously started working after adding logging to the constructor/on

Constructing ES8388 device...
Initalizing ES8388 codec...
Writing 0x50 to register 1
Writing 0x00 to register 2
Writing 0x08 to register 3
Writing 0x3c to register 4
Writing 0x00 to register 8
Writing 0x0c to register 12

Nathan 02/20/2023 12:21 AM

https://github.com/c22/toit-scratch/blob/main/ES8388_driver.toit updated

floitsch 02/20/2023 12:23 AM

Looked through it, and looks good.

floitsch 02/20/2023 12:23 AM

The print issue seems to have been a fluke, so I didn't expect to see anything bad...

Nathan 02/20/2023 12:23 AM

yes probably caffeine hadn't kicked in yet(edited)

Nathan 02/20/2023 01:18 AM

In case it's useful, I've transcribed the initialization of the driver here https://github.com/thaaraak/es8388/blob/master/src/es8388.cpp#L84-L120
into "plain english", this is mainly just for my own benefit

// https://github.com/thaaraak/es8388/blob/master/src/es8388.cpp#L84-L120

Writing 0x04 to register 25    // 0x04 mute/0x00 unmute&ramp;DAC unmute and disabled digital volume control soft ramp
Writing 0x50 to register 1    // normal all and power up all
Writing 0x00 to register 2
Writing 0x00 to register 8    // CODEC IN I2S SLAVE MODE
Writing 0xC0 to register 4    // disable DAC and disable Lout/Rout/1/2
Writing 0x12 to register 0    // Enfr=0,Play&Record Mode,(0x17-both of mic&paly)
Writing 0x18 to register 23    // 1a 0x18:16bit iis , 0x00:24
Writing 0x02 to register 24    // DACFsMode,SINGLE SPEED; DACFsRatio,256
Writing 0x00 to register 38    // 0x00 audio on LIN1&RIN1,  0x09 LIN2&RIN2
Writing 0x90 to register 39    // only left DAC to left mixer enable 0db
Writing 0x90 to register 42    // only right DAC to right mixer enable 0db
Writing 0x80 to register 43    // set internal ADC and DAC use the same LRCK clock, ADC LRCK as internal LRCK
Writing 0x00 to register 45    // vroi=0
Special handling for ADC/DAC volume on registers 16, 17, 26 and 27    // 0db
Special handling for enabled outputs on register 4
Writing 0xFF to register 3
Writing 0x11 to register 9    // MIC Left and Right channel PGA gain
Special handling for enabled inputs on register 10
Writing 0x02 to register 11
Writing 0x0d to register 12    // Left/Right data, Left/Right justified mode, Bits length, I2S format
Writing 0x02 to register 13    //ADCFsMode,singel SPEED,RATIO=256

//ALC for Microphone
Special handling for ADC/DAC volume on registers 16, 17, 26 and 27    // 0db
Writing 0x09 to register 3    //Power on ADC, Enable LIN&RIN, Power off MICBIAS, set int1lp to low power mode

(edited)

Nathan 02/20/2023 01:21 AM

my driver does a lot less, but I am trying to boil it down to only what's needed.. looking at this, perhaps setting volume levels is relevant(edited)

floitsch 02/20/2023 01:21 AM

Very useful.

Nathan 02/20/2023 01:28 AM

I'm also not too concerned with the DAC output right now as I'm still attempting to read the ADC before I tackle that part, so can probably ignore a lot of the DAC specific stuff for now...

Nathan 02/20/2023 03:36 AM

I changed tactics and tried to set the exact same register values (or as close as possible) as thaaraak's code above, but still getting zeroes out of the I2S bus

Initalizing ES8388 codec...
Writing 0x04 to register 25
Writing 0x50 to register 1
Writing 0x00 to register 2
Writing 0x00 to register 8
Writing 0xC0 to register 4
Writing 0x12 to register 0
Writing 0x18 to register 23
Writing 0x02 to register 24
Writing 0x00 to register 38
Writing 0x90 to register 39
Writing 0x90 to register 42
Writing 0x80 to register 43
Writing 0x00 to register 45
Writing 0x00 to register 16
Writing 0x00 to register 17
Writing 0x00 to register 26
Writing 0x00 to register 27
Writing 0x3F to register 4
Writing 0xFF to register 3
Writing 0x11 to register 9
Writing 0x00 to register 10
Writing 0x02 to register 11
Writing 0x0D to register 12
Writing 0x02 to register 13
Writing 0x09 to register 3

Nathan 02/20/2023 03:44 AM

Here's the code for that

https://github.com/c22/toit-scratch/blob/main/ES8388_driver_brute.toit

Back to being out of ideas

MikkelD 02/20/2023 06:16 AM

What I do at this point is to attach a logic analyser and test that there is a signal on the aux in, look at the i2s bus between codec and ESP32.

MikkelD 02/20/2023 08:08 AM

Also, I tend to not care to much about the initialization sequence until I have audio through the system. The initialization mostly is to avoid clicks and stuff in the beginning, so not important for proto types. I also tend to just write absolute values (not read and do the bitwise opertations) and then when I am done initializing, I read out all registers and verify by hand that it looks ok. There is a %b to give you bit patterns.

Nathan 02/20/2023 08:09 AM

I don't think I have a logic analyser, best I have is a multimeter

MikkelD 02/20/2023 08:09 AM

Multimeter is not going to be much help

Nathan 02/20/2023 08:09 AM

hehe

Nathan 02/20/2023 08:09 AM

and yeah the brute force driver is basically just writing the values directly to the register

Nathan 02/20/2023 08:10 AM

I think that I might have something to do with the i2s initialization wrong, perhaps..

MikkelD 02/20/2023 08:10 AM

"0b%(08b value)"

I think.

MikkelD 02/20/2023 08:11 AM

So, the I2S on the codec in slave mode needs only to set the CODEC in slave mode and set the word length.

Nathan 02/20/2023 08:11 AM

I could try getting it to work with the official ESP-ADF code, at least then i'd know i don't have a faulty part or something

MikkelD 02/20/2023 08:11 AM

Also, I am sure you have tried, but switching RX/TX in the ESP32 I2S port config.

Nathan 02/20/2023 08:11 AM

do I set the i2s bus itself to be in slave or master?

Nathan 02/20/2023 08:11 AM

i did try that

MikkelD 02/20/2023 08:13 AM

The I2S on the ESP32 needs to be in master (this is the default). This is set per I2S port, so they can be different for the two I2S ports. The I2S on the CODEC needs to be slave.

Nathan 02/20/2023 08:14 AM

hmm ok, I have that..

MikkelD 02/20/2023 08:20 AM

One thing I noticed, the LyraT has the MCLK connected on the CODEC, so you might want to add that on the ESP32 side.
Add --mclk=(gpio.Pin 0)

MikkelD 02/20/2023 08:21 AM

Actually, reading the ES8388 datasheet it needs the MCLK even in slave mode.

Nathan 02/20/2023 08:22 AM

interesting, OK I can give that a try

Nathan 02/20/2023 08:24 AM

wow.. you're a mad scientist! it worked!

Nathan 02/20/2023 08:24 AM

I'm just happy to see something other than zeroes!

Writing 32 bytes
#[0xf9, 0xfe, 0x51, 0xff, 0xf5, 0xfe, 0x51, 0xff, 0xf7, 0xfe, 0x59, 0xff, 0xfd, 0xfe, 0x4f, 0xff, 0xf9, 0xfe, 0x4d, 0xff, 0x05, 0xff, 0x51, 0xff, 0xf9, 0xfe, 0x5b, 0xff, 0xfd, 0xfe, 0x55, 0xff]

(edited)

MikkelD 02/20/2023 08:25 AM

Awesome.

MikkelD 02/20/2023 08:26 AM

My bad, because the CODEC we use does not need the MCLK, but uses just the BCLK. It is apparently different from codec to codec.

Nathan 02/20/2023 08:26 AM

that works with the brute force driver.. but now it gives me some hope to continue working on my original driver

Nathan 02/20/2023 08:27 AM

totally fine I had a feeling it was something so stupidly obvious that I must be missing...

Nathan 02/20/2023 08:28 AM

I will work on it a bit more this week and hopefully share something more polished soon

floitsch

That's a common problem with i2c. When reading from i2c one takes the ID and shifts it. When writing one shifts and adds one. The datasheets often just say that the ID is the shift...

Fedex 02/24/2023 01:15 PM

Now I understand why the bus scan returned me something else! Thanks for explaining it.

122 messages in total