• 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
NIOT-E-NPI3-EN Excessive CPU Usage
#1
Hello,
We have two NIOT-E-NPI3-EN units that we are currently evaluating internally for a host for a software product we are currently developing. Both units carry part #1321.510. Both units have been configured identically with both the internal gateway manager settings and with the current Docker configuration.

But one of the units is showing a constantly higher CPU usage that has brought the web server down to unusable performance levels and cannot be used. The poor performing unit has a CPU usage of 26 to 32%, even in an idle state, whereas the unit that functions correctly has a CPU usage of 3% in an idle state.

When evaluating the downloaded syslog on the poor performing unit, we noticed the log file is excessive in size, over 630MB in size compared to just a few MB's in size for the properly functioning unit on any given day sense we started testing. When evaluating the log, we have found over 341 entries in the first minute of operations. The vast majority of the entrees seem to be system related and we cannot determine their meaning. I included an attachment of the first minute of the Syslog for startup reference.

The date code on the inoperable unit is 19/38 and the date code on the properly functioning unit is 20/16.

Can someone please advise what we can do to continue to try to trouble shoot this unit or if there is another direction we should take to remedy this units performance? Please let me know if there is any other information I can provide as well?

Thank you,
Mark


Attached Files
.txt   Hilscher Net pi Core 3 Syslog 11-18-20.txt (Size: 54.32 KB / Downloads: 3)
  Reply
#2
Try recovering the "unstable" unit with an USB stick as described in the FAQ list of netPI here https://www.netiot.com/?id=2427 and report what happens with the unit afterwards. The procedure will be formatting the SD card and resetting the whole unit and parameter set to factory default.

Based on the log file your sent and the unit ID logged I see that this unit has been produced 2019-09-25. This unit is using an SD card from company XMORE while all units produced after month 10/2019 are using SD cards from company Sandisk. This is the only hardware difference between both units. But these two cards are not that different to produce a CPU load difference of 30%. Usual is a value of 2-5% CPU load.

What I have seen myself in the past with one of my older units <10/2020 was a similar effect and it was gone using the USB update stick procedure. My explanation was that something was wrong with the SD card or file system access. This is why I am recommending to you to perform a recovery.

Thx
Armin
You never fail until you stop trying.“, Albert Einstein (1879 - 1955)

  Reply
#3
(November-18th-2020, 06:32 PM)Armin@netPI Wrote: Try recovering the "unstable" unit with an USB stick as described in the FAQ list of netPI here https://www.netiot.com/?id=2427 and report what happens with the unit afterwards. The procedure will be formatting the SD card and resetting the whole unit and parameter set to factory default.

Based on the log file your sent and the unit ID logged I see that this unit has been produced 2019-09-25. This unit is using an SD card from company XMORE while all units produced after month 10/2019 are using SD cards from company Sandisk. This is the only hardware difference between both units. But these two cards are not that different to produce a CPU load difference of 30%. Usual is a value of 2-5% CPU load.

What I have seen myself in the past with one of my older units <10/2020 was a similar effect and it was gone using the USB update stick procedure. My explanation was that something was wrong with the SD card or file system access. This is why I am recommending to you to perform a recovery.

Thx
Armin

Hello Armin,
Thank you for responding. I have followed the steps you indicated and the unit seems to have responded correctly. I was able to witness the recovery program on the local terminal monitor and when it was finished and I rebooted the unit, it appeared to reboot correctly as the local machine.

Unfortunately now the problem is that I cannot login into the unit using the default address of https://NTB827EB8B0E4D as noted on the box it came in. I double checked my network settings and my adapter is set to DHCP. I checked the adapter status and it is showing an assigned Connection-specific DNS Suffix:
Description: Realtek RTL8150 USB 10/100 Fast Ethernet Adapter
DHCP Enabled: Yes
Autoconfiguration IPv4 Address: 169.254.65.8
IPv4 Subnet Mask: 255.255.0.0
IPv4 Default Gateway:
IPv4 DNS Server:
IPv4 WINS Server:
NetBIOS over Tcpip Enabled: Yes

I am assuming as it has been assigned an IP address, the units DHCP server is working and communicating. But as I cannot raise a local terminal, I cannot check the local ipcondifg settings. I also have the unit cabled directly into my network adapter on my PC, there is no router or switch in between. The response in my browser window is "ntb827eb8b0e4d’s server IP address could not be found." and the error of "ERR_NAME_NOT_RESOLVED". I would also assume the unit would still carry its default access address? Or is there another way to possible access it?

Thank you,
Mark
  Reply
#4
Well even after the update the Hostname printed on the label will remain the same. Of course you can change it afterwards once you logged in to the device successfully.

The usual way (cause I have several Hilscher devices running in my network) I am doing it is using the Windows File Explorer and the option "network" that lists you all printers, cameras, NAS server etc. and also our netPIs.

   

Try this one and I am pretty sure you will find it there.
You never fail until you stop trying.“, Albert Einstein (1879 - 1955)

  Reply
#5
Back to you question about a different way to get access to this unit: no, there isn't any. after the update the device is operating in factory default.

Have you waited long enough after the reboot and update to give the unit the chance to recover?
You never fail until you stop trying.“, Albert Einstein (1879 - 1955)

  Reply
#6
Make sure before booting the device that you have removed the USB update stick. :-)
You never fail until you stop trying.“, Albert Einstein (1879 - 1955)

  Reply
#7
(November-19th-2020, 06:34 PM)Armin@netPI Wrote: Make sure before booting the device that you have removed the USB update stick. :-)

Hello Armin,
Yes, I sometimes forgot to remove the card myself. Smile
I have tried searching out the unit in IE using Network, but it does not show up. (Interestingly, I can find the working unit this way when I look for it, very strange.)

I'm not sure how else to try to find it? Is there another method I can try?
Thanks,
Mark
  Reply
#8
You say IE = Internet Explorer ... I mean the Windows File Explorer. This program has the "network" seach function only.

You said you have two units ... how about the working unit? Do you see this unit in the Windows File Explorer at least?

How about sending a photo of the HDMI output of the non working unit when it has finished booting. Usually a netPI stops at a certain point it outputting logging data and then it should be available on the network as expected. I just want to see if the unit boots correctly to the end.

Thx
Armin
You never fail until you stop trying.“, Albert Einstein (1879 - 1955)

  Reply
#9
Hi Armin,
I apologize, I have been on holiday the last few days.
I used the wrong name, but yes, I did use Windows File Explorer's Network search.

I first tried setting up and detecting the good Net pi unit using its DHCP server and I was able to find it in the Network window of File Explorer with no problems. It had used the DHCP range I had changed the configuration to and provided addresses on both machines I used adapters.

With the exact same network adapter settings, I then disabled both machines Ethernet adapters, then disconnected the working unit from the internal network (which consists of just a un-managed switch, the PC's and the Net pi units connected at any one time) and I then I re-enabled both network adapters on the PC's. After comparing the adapter status's compared to the working unit, it does not appear that the Net pi unit is assigning correct IP settings to the PC adapters as shown in the two attached images.

I also tried this exact same connection method as above using an Industrial workstation running Ubuntu 20 OS and it also produced the same results. I was able to get an assigned IP address and locate the unit in the "Other Locations" tab of the Files window, and I was able to access the good unit. But on the bad unit the network adapter monitor was showing that it was not obtaining an IP address and after 3 tries, it would disable itself and of course I was not able to locate the unit in the Files window. I have also attached an image of that units settings and results.

Is there anything else you can think of to access this Net pi unit?

Thank you
Mark

Hello Armin,
Sorry, I forgot to post the startup screens for the bad unit I am calling B0E4D. I tried to capture the scrolling several times, but was only able to get 3 screen captures that were legible. Please see attached.

Thank you,
Mark


Attached Files Thumbnail(s)
                       
  Reply
#10
Hi Mark,

thank you for sending me the "whole "boot sequence. The very interesting part is just the last one ... where the unit finally ends outputting log data. This usually shows me whether or not a device booted to the end successfully.

Based on your last screen shot ... I would say the unit boots to the end correctly. I am pretty sure your good unit would output the very same data.

Nevertheless ... the good unit is available via Windows file explorer ... the bad one not. I see no possibility to get it back to operation right now and to be honest I have no explanation what happens to it ... except the Ethernet RJ45 port has a HW malfunction. In this case the unit needs anyway sent back for repair. So you see the Link/Act LEDs of this port static on/flashing on traffic. This is indeed my last option to look at.

Are you really sure the update stick used with this unit run to the end ... watched the update sequence over HDMI for a check? It seems this unit has not the default DHCP setting and is using more a static IP address configuration rather.

Well if all this does not help this unit needs to be sent back to us.
You never fail until you stop trying.“, Albert Einstein (1879 - 1955)

  Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  NIOT-E-TPI51-EN-RE Firmware 1.2.2.0 not functional COswald 3 3,264 February-4th-2022, 06:20 PM
Last Post: Armin@netPI
  NPIX expansion module NIOT-E-NPIX-4DI4DO 3D drawing Armin@netPI 0 2,017 April-20th-2020, 05:49 AM
Last Post: Armin@netPI
Wink SD card on NIOT-E-NPI3-51-EN-RE Lingyue 3 4,220 March-6th-2020, 07:34 AM
Last Post: Lingyue
  NIOT-E-NPIX-EVA EPROM milesie 1 2,465 February-24th-2020, 06:32 PM
Last Post: Armin@netPI
  NIOT-E-NPIX-RCAN availability andreash 4 4,366 April-26th-2019, 01:25 PM
Last Post: Armin@netPI

Forum Jump:


Users browsing this thread: 3 Guest(s)