• 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
cifX drivers EtherCAT read/write performance with NetHat over STM32
#1
Hello,

I've started experimenting with EtherCAT and NetHAT over STM32MP157C-DK2 board. I measured the xChannelIORead and xChannelIOWrite performances and I'm obtaining results more than 2ms for each, between 4-5ms for both with a large amount of jitter. My embedded Linux is patched with PREEMPT_RT and the measurements were done inside a high priority RT thread.
I was expecting to get latencies < 1 ms for real time with EtherCAT and I was wondering if these measurements are expected on the NetHAT.

This is the function I used for measuring time:

Code:
uint64_t getTimeNanoseconds() {
    struct timespec ts;
    clock_gettime(CLOCK_MONOTONIC, &ts);
    return (uint64_t)(ts.tv_sec) * 1000000000 + (uint64_t)(ts.tv_nsec);
}

How the measurement is done:

Code:
    while(counter < 10000)
    {
        while( CIFX_DEV_NO_COM_FLAG == (lRet = xChannelBusState( hChannel, CIFX_BUS_STATE_ON, &ulState, 10))){
           printf("Waiting for Bus communication!\r\n");
        }

        abSendData[1] = !abSendData[1];    


        startTime = getTimeNanoseconds();
        if(CIFX_NO_ERROR == (lRet = xChannelIORead(hChannel, 0, 0, sizeof(abRecvData), abRecvData, 10))){
            tr_r = getTimeNanoseconds() - startTime;
            sum_r += tr_r;
            rTimes[counter] = tr_r;
        }
        else {
            printf("Error reading \n");
            return NULL;
        }


        startTime = getTimeNanoseconds();

        if(CIFX_NO_ERROR == (lRet = xChannelIOWrite(hChannel, 0, 0, sizeof(abSendData), abSendData, 10))){
            tr_wr = getTimeNanoseconds() - startTime;
            sum_wr += tr_wr;
            wTimes[counter] = tr_wr;
        }
        else {
            printf("Error writing \n");
            return NULL;
        }

        counter++;    

        printf("R | Time : %llu Average : %llu | W | Time : %llu Average : %llu |\n", tr_r, sum_r / counter, tr_wr, sum_wr / counter);    


        usleep(cycleTime);        
    }


I obtain different performances depending on "cycleTime" - time between each read/write call -  I have much more jitter with cycle time < 1ms than for example 10ms. I also have different performances depending on when I execute, I can execute same code twice and get different average latencies. 

The cycle time of the master is 200us for sending/receiving process data.

I'm not quite familiar with real time protocol programming and I would like to know if these measurements are expected if not I'm curious to know the cause of these latencies.

 Thank you,
  Reply
#2
If you are using the STM32MP157C-DK2 board ... how do you connect a netHAT made for Raspberry Pi with 40 pin GPIO header to this ST main board? I suppose I did miss something in your setup?
You never fail until you stop trying.“, Albert Einstein (1879 - 1955)

  Reply
#3
The STM32MP157C-DK2 board has a 40 pin GPIO port compatible with Raspberry Pi shields so I just have to plug it in.
  Reply
#4
Interesting ... never heard about such a compatible pinning. But which netHAT driver do you use in conjunction with the ST processor? Is the 100% the same that we provide for the Raspberry Pi CPU?
You never fail until you stop trying.“, Albert Einstein (1879 - 1955)

  Reply
#5
Yes it is the same driver. I installed it manually by copy pasting because I couldn't open .deb file (package architecture wasn't working on mine)
I use my own Linux built with Yocto. And for information I also set up a kernel flag config_uio_netx=y I don't know if it's for any use for the netHAT.
For the pinning I simply had to plug it in and configure through CubeMX to activate the spi at 25mhz and make sure pins used by NetHAT are available (reset state).
  Reply
#6
Ok understood. Here is the information I can give to you concerning netHAT. netHAT has never been designed to support jitterless and time equidistant IO data transmissions across the SPI interface as it was designed from the beginning with Hilscher's PCIe PC cards using a netX100 chip and DMA transportation of data from the internal netX100 RAM directly to the PC's RAM and back. This hardware supported transmissions go nearly jitterless and in some usecs.

This is of course not possible across SPI. But here are some information where maybe the performance is getting improved.

1. The driver we are providing for netHAT is meanwhile 4 years old. We have improved the driver, not for netHAT, but for netPI. An updated version you find here https://github.com/HilscherAutomation/ne...ter/driver
2. The new driver supports Interrupt driven mode. Since the old driver polls for data the new one is getting an interrupt whenever the netX is new information for the driver.
3. Using Interrupt needs the corresponding GPIO24  to be configured as input. This is done on Raspberry Pi using the code


Code:
if [[ ! -e "/sys/class/gpio/gpio24" ]]; then
   echo 24 > /sys/class/gpio/export
 
fi
 echo rising > /sys/class/gpio/gpio24/edge
 echo in > /sys/class/gpio/gpio24/direction
 echo 1 > /sys/class/gpio/gpio24/active_low


4. Since the config file of the driver has no interrupt enabled by default you need to activate the interrupt mode in the driver by writing corresponding value the the file

Code:
echo "Irq=/sys/class/gpio/gpio24/value" >> /opt/cifx/plugins/netx-spm/config0

5. Now there is two other configuration values next to "Irq=..." that can be confgured and more or less secret

The first is

PHP Code:
Irqprio=xxx 
which is configuing as integer value the priority of the ISR handler thread (0 = default, as priority of hte calling thread). See Linux man pages pthread_attr_setschedparam.

the other is
Code:
irqsched=xxx

which is configuring as string the value 'fifo' the method FIFO scheduling (see SCHED_FIFO -> irqprio 1...99, while 1 is low and 99 high) or the value 'rr' real-time scheduling (see SCHED_RR -> irqprio 1..99, while 1 is low and 99 high)

And yes the first parameter starts with a capital letter 'I' and the second a small letter 'i". Don't know why.

This information is given to you as is and works with original Raspberry Pi. If it works with ST I can't guarantee.

Thx
Armin
You never fail until you stop trying.“, Albert Einstein (1879 - 1955)

  Reply
#7
Thanks for the information.
I have updated the driver and setup the IRQ on GPIO18. The SPM_DIRQ on my NetHAT is GPIO18 and not GPIO24 which is used for spi chip select.
If I activate the IRQ I have better performance with 1 to 2ms less latency. But I still have around 3-4ms total W/R latency depending on frequency of call.

I'd be interested to get a confirmation that these performances are expected or abnormally high. Are these performances close to what other users get on the RaspberryPi ? I want to be sure that this is not a problem coming from my code or my STM32 configuration.
  Reply
#8
Oh yes indeed, I looked into the schematics of netHAT. netHAT uses GPIO18 as IRQ and not GPIO 24 as with netPI. Thank you for clearance.

I know of a customer that is using real-time Linux kernel with our netPI device - which is mainly a Raspberry Pi 3B with a netX chip inside connected over the same SPI interface - and I could ask him about the performance improvement. He was also using the old driver first and I gave him the same information as I gave it to you ... and after that I have never heard from him again if something has improved or not. I will ask him about his performance.

I personally have never ever measured any latency and jitter since for 99.99% of all applications I know of it is not of any interest.

I will be back to you.

Thx
Armin
You never fail until you stop trying.“, Albert Einstein (1879 - 1955)

  Reply
#9
There is one additional note from my side concerning the serial port memory (SPM) that is integral part of the netX chip. This "device" is realized in hardware, so it is working always in the same manner and speed. It shifts the acquired data as a stream one byte after the other over the SPI bus in the frequency you configured the SPI clock with. There is no gap in between the single bytes. So with this logic I do not expect any jitter and unpredictable delays.

But since the SPM is designed just to transport data to and from an internal netX memory block 1:1 in hardware there is on the other hand software running in the chip that is filling the memory with values or is reading from it all in accordance to two handshake memory cells that are integral part of the mmory block serving as sychronization mechanism.

This memory block that can be accessed via the SPM is not the buffer the netX chip will copy the latest received EtherCAT master data to or read from it as data for the master . For this there are two times extra triple buffers. Whenever the SPM host system (your app) requests for IO data the softwre copies the data from the latest filled triple buffer to the SPM memory and indicates this in the handshake flags. This software driven meachnism is of couse jitter influenced since it is software. Also your host software comes with jitter. But I hardly can imagine this all would lead to a jtter of 4 msec.

If you say up to 4 msecs jitter. What exactly do you mean with that? What timings do you measure?
You never fail until you stop trying.“, Albert Einstein (1879 - 1955)

  Reply
#10
I measure the time consumed by the calls 'xChannelIORead' and 'xChannelIOWrite', you can see it on the code I posted. And the average of both are 2ms+2ms=4ms. It's not the jitter but the average time consumed by calling read and write. But there are also regular jitters of several ms on top of that.
  Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Using Powerlink firmware with netHAT and config nxd thp9 3 505 August-25th-2020, 01:49 PM
Last Post: Armin@netPI
  NetHAT SPI configuration thp9 1 699 June-10th-2020, 02:05 PM
Last Post: Armin@netPI
  How can i set a EtherCAT node in netHAT 52-RTE? JGLee 6 856 June-9th-2020, 12:38 PM
Last Post: Armin@netPI
  netHAT as Ethernet/IP Adapter with Codesys lstiegler 11 1,060 May-26th-2020, 06:40 PM
Last Post: Armin@netPI
  netHat configuration via config packets pitau 6 713 May-25th-2020, 02:21 PM
Last Post: Armin@netPI
  NetHAT temperature question thp9 1 650 May-14th-2020, 09:21 PM
Last Post: Armin@netPI
  Ethercat Master TwinCATversion? MGharat 1 163 September-19th-2019, 06:38 AM
Last Post: Armin@netPI
  Using netHAT drivers with custom linux (not Raspbian) FrankS 5 737 September-11th-2019, 06:16 AM
Last Post: Armin@netPI
  Use netPi firmware on netHAT zz_zesys 1 962 September-30th-2018, 08:04 PM
Last Post: Armin@netPI
  NetHat Labview richielfc 1 785 August-10th-2018, 12:08 PM
Last Post: Simon@netHAT

Forum Jump:


Users browsing this thread: 1 Guest(s)