Mobile Development: A remote cpu monitor and cpu usage analysis
cpumon2 and cpumonRcv
Although this project is not yet finished, the main function works very well: viewing cpu usage remotely and capture data to database for later analysis.
Some of the ideas and code is inspired by a cpu usage article at http://www.codeproject.com/Articles/159461/Mobile-Processor-Usage. Further on I got more questions the last days like “Why is app x running slow?” or “When I start that app, the system gets slow and taskmanager shows a high cpu usage.”.
Here are two tools to capture cpu usage of a windows mobile device remotely via a TCP/IP connection.
Before you read on, the code is by far not perfect and there are still many improvements possible. But if you need a working remote cpumon, here we go…
cpumon on device
Why another cpumon or taskmanager like app? Simply as you may test an application without having to switch between the app and taskmanager to see cpu usage. Switching back and for may also interrupt the workflow in your app.
cpumon creates snapshot of all processes and threads periodically including there used user and kernel times. On windows ce, the kernel times are always zero, but that may change in future and so I included that in the statistics info too.
First, cpumon creates a snapshot of all processes with there process ID’s. Then another snapshot of all threads is created with there owner ProcID, thread ID and usage times. There is a delay between two snpashots further called duration. The threads are assigned to the processes and there times (user and kernel) are summarized. This statistics snapshot is put in a queue. This will repeat all 3 seconds. At the end of a statistics set, another thread is released that dequeues the data and sends it using an UDP socket. I decided to use UDP broadcast as this is fast and requires no client/server connection. At the end of a set, one <EOT> packet is sent to let the host know, that a set has been transfered.
The statistic data is also shown in the GUI of cpumon, but this is only for reference.
cpumonRcv
cpumonRcv is a desktop csharp app that listens for UDP broadcasts of cpumon. It converts the received bytes back to a statistic set of processes and threads and en-queues the data until an EOT packet has arrived. Then the data is saved to a local database and can be used for later analyzes. Additionally every new statistic data packet will update a live view.
The code
On the device
A process has an exe name and a list of threads. Every thread is captured with the time it has spend in user and kernel space. The threads and processes are recorded with a timestamp and the duration of two consecutive snapshots. If you sum the user times of all threads of a process and divide that by the duration and multiply with 100 you get the percentage the process has used of the processors time.
The above is reflected in a thread and a process class. Further on, these classes include a conversion from/to byte arrays. These byte arrays build the packet that is send via UDP broadcasts.
The transmitter (cpumon) and receiver share the same code for process and thread class. So the receiver can easily transform byte packets back to processes and threads.
The main class is ProcInfo. It starts the snapshot (usage) thread and the socket thread.
public ProcInfo()
{
statisticsTimes = new Dictionary<string, ProcessStatistics.process_statistics>();
eventEnableCapture = new AutoResetEvent(true);
eventEnableSend = new AutoResetEvent(false);
//procStatsQueue = new Queue<ProcessStatistics.process_statistics>();
procStatsQueueBytes = new Queue<byte[]>();
myThreadSocket = new Thread(socketThread);
myThreadSocket.Start();
myThread = new Thread(usageThread);
myThread.Start();
}
The snapshot thread captures a snapshot of processes and threads and adds the data to a dictionary and en-queues it for the socket thread. I use a dictionary for the process stats as this automatically inserts or updates process data. So I do not need to look first, if a process is already known or not and if I have to update existing data or add new data. Another reason is that the code has to look up a the thread list for existing data (TryGetValue) to build the new stats.
/// <summary>
/// build thread and process list periodically and fire update event and enqueue results for the socket thread
/// </summary>
void usageThread()
{
try
{
int interval = 3000;
uint start = Process.GetTickCount();
Dictionary<uint, thread> old_thread_List;// = Process.GetThreadList();
string exeFile = Process.exefile;
//read all processes
Dictionary<uint, process> ProcList = Process.getProcessNameList();
DateTime dtCurrent = DateTime.Now;
//######### var declarations
Dictionary<uint, thread> new_ThreadList;
uint duration;
long system_total;
long user_total, kernel_total; //total process spend in user/kernel
long thread_user, thread_kernel; //times the thread spend in user/kernel
DWORD dwProc;
float user_percent;
float kernel_percent;
ProcessStatistics.process_usage usage;
ProcessStatistics.process_statistics stats = null;
string sProcessName = "";
List<thread> processThreadList = new List<thread>();
//extended list
List<threadStatistic> processThreadStatsList = new List<threadStatistic>(); //to store thread stats
while (!bStopMainThread)
{
eventEnableCapture.WaitOne();
old_thread_List = Process.GetThreadList(); //build a list of threads with user and kernel times
System.Threading.Thread.Sleep(interval);
//get a new thread list
new_ThreadList = Process.GetThreadList(); //build another list of threads with user and kernel times, to compare
duration = Process.GetTickCount() - start;
ProcList = Process.getProcessNameList(); //update process list
dtCurrent = DateTime.Now;
system_total = 0;
statisticsTimes.Clear();
//look thru all processes
foreach (KeyValuePair<uint, process> p2 in ProcList)
{
//empty the process's thread list
processThreadList=new List<thread>();
processThreadStatsList = new List<threadStatistic>();
user_total = 0; //hold sum of thread user times for a process
kernel_total = 0; //hold sum of thread kernel times for a process
sProcessName = p2.Value.sName;
//SUM over all threads with that ProcID
dwProc = p2.Value.dwProcID;
foreach (KeyValuePair<uint, thread> kpNew in new_ThreadList)
{
thread_user = 0;
thread_kernel = 0;
//if the thread belongs to the process
if (kpNew.Value.dwOwnerProcID == dwProc)
{
//is there an old thread entry we can use to calc?
thread threadOld;
if (old_thread_List.TryGetValue(kpNew.Value.dwThreadID, out threadOld))
{
thread_user=Process.GetThreadTick(kpNew.Value.thread_times.user) - Process.GetThreadTick(old_thread_List[kpNew.Value.dwThreadID].thread_times.user);
user_total += thread_user;
thread_kernel =Process.GetThreadTick(kpNew.Value.thread_times.kernel) - Process.GetThreadTick(old_thread_List[kpNew.Value.dwThreadID].thread_times.kernel);
kernel_total += thread_kernel;
}
//simple list
thread threadsOfProcess = new thread(kpNew.Value.dwOwnerProcID, kpNew.Value.dwThreadID, kpNew.Value.thread_times);
processThreadList.Add(threadsOfProcess);
//extended list
threadStatistic threadStats =
new threadStatistic(
kpNew.Value.dwOwnerProcID,
kpNew.Value.dwThreadID,
new threadtimes(thread_user, thread_kernel),
duration,
dtCurrent.Ticks);
processThreadStatsList.Add(threadStats);
}//if dwProcID matches
}
//end of sum for process
user_percent = (float)user_total / (float)duration * 100f;
kernel_percent = (float)kernel_total / (float)duration * 100f;
system_total = user_total + kernel_total;
// update the statistics with this process' info
usage = new ProcessStatistics.process_usage(kernel_total, user_total);
// update process statistics
stats = new ProcessStatistics.process_statistics(p2.Value.dwProcID, p2.Value.sName, usage, dtCurrent.Ticks, duration, processThreadStatsList);
//add or update the proc stats
if (exeFile != p2.Value.sName || bIncludeMySelf)
{
statisticsTimes[p2.Value.sName] = stats;
procStatsQueueBytes.Enqueue(stats.ToByte());
}
start = Process.GetTickCount();
}//foreach process
onUpdateHandler(new ProcessStatsEventArgs(statisticsTimes, duration));
procStatsQueueBytes.Enqueue(ByteHelper.endOfTransferBytes);
((AutoResetEvent)eventEnableSend).Set();
}//while true
}
catch (ThreadAbortException ex)
{
System.Diagnostics.Debug.WriteLine("ThreadAbortException: usageThread(): " + ex.Message);
}
catch (Exception ex)
{
System.Diagnostics.Debug.WriteLine("Exception: usageThread(): " + ex.Message);
}
System.Diagnostics.Debug.WriteLine("Thread ENDED");
}
The other thread, the socket thread, looks like this:
/// <summary>
/// send enqueued objects via UDP broadcast
/// </summary>
void socketThread()
{
System.Diagnostics.Debug.WriteLine("Entering socketThread ...");
try
{
const int ProtocolPort = 3001;
sendSocket = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
sendSocket.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.Broadcast, 1);
sendSocket.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.SendBuffer, 32768);
IPAddress sendTo = IPAddress.Broadcast;// IPAddress.Parse("192.168.128.255"); //local broadcast
EndPoint sendEndPoint = new IPEndPoint(sendTo, ProtocolPort);
//UdpClient udpC = new UdpClient("255.255.255.255", 1111);
System.Diagnostics.Debug.WriteLine("Socket ready to send");
while (!bStopSocketThread)
{
//block until released by capture
eventEnableSend.WaitOne();
lock (lockQueue)
{
//if (procStatsQueue.Count > 0)
while (procStatsQueueBytes.Count > 0)
{
//ProcessStatistics.process_statistics pStat = procStatsQueue.Dequeue();
//byte[] buf = pStat.ToByte();
byte[] buf = procStatsQueueBytes.Dequeue();
if (ByteHelper.isEndOfTransfer(buf))
System.Diagnostics.Debug.WriteLine("sending <EOT>");
sendSocket.SendTo(buf, buf.Length, SocketFlags.None, sendEndPoint);
//System.Diagnostics.Debug.WriteLine("Socket send " + buf.Length.ToString() + " bytes");
//System.Diagnostics.Debug.WriteLine(pStat.dumpStatistics());
System.Threading.Thread.Sleep(2);
}
}
((AutoResetEvent)eventEnableCapture).Set();
}
}
catch (ThreadAbortException ex)
{
System.Diagnostics.Debug.WriteLine("ThreadAbortException: socketThread(): " + ex.Message);
}
catch (Exception ex)
{
System.Diagnostics.Debug.WriteLine("Exception: socketThread(): " + ex.Message);
}
System.Diagnostics.Debug.WriteLine("socketThread ENDED");
}
After setting up the socket for port 3001 the thread waits for the event eventEnableSend. Then it starts the transfer of all en-queued packets. The last packet in the queue should be the <EOT> packet, which has been en-queued by the usageThread. The usageThread is waiting during the transfer of the packets and will be released by the socketThread with setting the event eventEnableCapture.
As we can only send bytes thru the socket, the process statistics class include functions to be converted to or from byte arrays. Here is a snippet of the process_statistics class to show you how I did implement this.
public byte[] ToByte()
{
List<byte> buf = new List<byte>();
//string length
Int16 bLen = (Int16)System.Text.Encoding.UTF8.GetByteCount(sName);
buf.AddRange(BitConverter.GetBytes(bLen));
byte[] bName = System.Text.Encoding.UTF8.GetBytes(sName);
//string as byte[]
buf.AddRange(bName);
buf.AddRange(BitConverter.GetBytes(dateTime));
buf.AddRange(BitConverter.GetBytes(duration));
buf.AddRange(procUsage.ToByte());
buf.AddRange(BitConverter.GetBytes(processID));
//list count
Int16 iCnt = (Int16) ThreadStatList.Count;
threadStatistic[] threadsArray = ThreadStatList.ToArray();
buf.AddRange(BitConverter.GetBytes(iCnt));
//now add the threads of the list
foreach (threadStatistic th in threadsArray)
buf.AddRange(th.ToByte());
return buf.ToArray();
}
The packet (byte array) can vary in size as a string (the process name) has to be encoded and decoded. So the function first encodes the string length and then the string.
and here a function to get the object back from a byte array.
public process_statistics FromByte(byte[] buf)
{
int offset = 0;
Int16 bLen = BitConverter.ToInt16(buf, 0); //2 bytes
offset += sizeof(System.Int16);
if (bLen > 0)
this.sName = System.Text.Encoding.UTF8.GetString(buf, offset, bLen);
offset += bLen;
this.dateTime = BitConverter.ToInt64(buf, offset);
offset += sizeof(System.Int64);
this.duration = BitConverter.ToUInt32(buf, offset);
offset += sizeof(System.Int32);
this.procUsage = new process_usage(ref buf, ref offset);
//offset = offset; //has been changed by process_usage
this.processID = BitConverter.ToUInt32(buf, offset);
offset += sizeof(System.UInt32);
//how many thtreads are in the byte stream
Int16 iCnt = BitConverter.ToInt16(buf, offset);
offset += sizeof(System.Int16);
//start reading the threads
List<threadStatistic> thList = new List<threadStatistic>();
for (int x = 0; x < iCnt; x++)
{
threadStatistic th = new threadStatistic(buf, ref offset);
thList.Add(th);
}
this.ThreadStatList = thList;
return this;
}
Here the string has to be decoded and so first the length has to be read and then the string.
On the PC
cpumonRcv runs on a PC in the same subnet as the device is running cpumon. The receiver code listens for packets coming on UPD port 3001 and then re-assembles the byte to process statistics. These stats are saved to a local database (currently sqlite) and are send to the GUI for a live view.
The main function is the socket server thread code.
public void StartReceive()
{
receiveSocket = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
bindEndPoint = new IPEndPoint(IPAddress.Any, 3001);
recBuffer = new byte[maxBuffer];
receiveSocket.Bind(bindEndPoint);
receiveSocket.BeginReceiveFrom(recBuffer, 0, recBuffer.Length, SocketFlags.None, ref bindEndPoint, new AsyncCallback(MessageReceivedCallback), (object)this);
}
You see, that is not really a thread. Instead of polling the socket for received bytes within a thread, the code uses the asynchronous BeginReceiveFrom socket call.
The callback will be called whenever there are bytes received. As a packet always holds a complete statistic process data, the GUI can be updated after each received packet. But to be fast, the GUI is only updated when a <EOT> packet has arrived. The other stats data are delivered to the GUI thread, where they are recorded for later use and directly update the GUI.
When a packet has received, we must call BeginReceiveFrom again to be ready for the next packet.
void MessageReceivedCallback(IAsyncResult result)
{
EndPoint remoteEndPoint = new IPEndPoint(0, 0);
try
{
//all data should fit in one package!
int bytesRead = receiveSocket.EndReceiveFrom(result, ref remoteEndPoint);
byte[] bData = new byte[bytesRead];
Array.Copy(recBuffer, bData, bytesRead);
if (ByteHelper.isEndOfTransfer(bData))
updateEndOfTransfer();// end of transfer
else
{
ProcessStatistics.process_statistics stats = new ProcessStatistics.process_statistics(bData);
//System.Diagnostics.Debug.WriteLine( stats.dumpStatistics() );
updateStatus(stats);
}
}
catch (SocketException e)
{
System.Diagnostics.Debug.WriteLine(String.Format("MessageReceivedCallback SocketException: {0} {1}", e.ErrorCode, e.Message));
}
catch (Exception e)
{
System.Diagnostics.Debug.WriteLine(String.Format("MessageReceivedCallback Exception: {0}", e.Message));
}
try
{
//ready to receive next packet
receiveSocket.BeginReceiveFrom(recBuffer, 0, recBuffer.Length, SocketFlags.None, ref bindEndPoint, new AsyncCallback(MessageReceivedCallback), (object)this);
}
catch (Exception) { }
}
I placed the data capturing inside the GUI thread. The live view will be updated and a queue is filled with the data.
delegate void addDataCallback(ProcessStatistics.process_statistics procStats);
void addData(ProcessStatistics.process_statistics procStats)
{
if (this.dataGridView1.InvokeRequired)
{
addDataCallback d = new addDataCallback(addData);
this.Invoke(d, new object[] { procStats });
}
else
{
//enqueue data to be saved to sqlite
dataQueue.Enqueue(procStats);
//dataAccess.addSqlData(procStats);
dataGridView1.SuspendLayout();
//dtProcesses.Rows.Clear();
dataAccess.addData(procStats);
//dataGridView1.Refresh();
dataGridView1.ResumeLayout();
//release queue data
dataAccess.waitHandle.Set();
//object[] o = new object[7]{ procUsage.procStatistics. .procStatistics. [i].sApp, eventEntries[i].sArg, eventEntries[i].sEvent,
// eventEntries[i].sStartTime, eventEntries[i].sEndTime, eventEntries[i].sType, eventEntries[i].sHandle };
}
}
A queue is used for the background task that saves the data to a file. Saving will need some time and should not interfer with the live view or the data receive function.
Queue<System.Process.ProcessStatistics.process_statistics> dataQueue; Thread myDataThread; public EventWaitHandle waitHandle;
The data is ‘written’ to the local database using a separate thread.
void dataAddThread()
{
try
{
while (true)
{
waitHandle.WaitOne();
if (dataQueue.Count > 10)
{
while (dataQueue.Count > 0)
{
System.Process.ProcessStatistics.process_statistics procStats = dataQueue.Dequeue();
addSqlData(procStats);
}
}
Thread.Sleep(10);
}
}
catch (ThreadAbortException ex)
{
System.Diagnostics.Debug.WriteLine("Exception: " + ex.Message);
}
catch (Exception ex)
{
System.Diagnostics.Debug.WriteLine("Exception: " + ex.Message);
}
}
The receiver code has some time (3 seconds) to save the data between the send of two full statistic sets.
The live view shows a list of running processes in a datagridview and a small line plot of the usage times of the selected process.
There is another data view that is not yet finished. The detail view shows a list of captured stats with the processes and the timestamp. If you click a line, the threads stats of the process are displayed with there usage. The detail view shows the saved data, where the main view shows live data.
Export and excel analysis
Now I have also finished export functions. From the detail form you can use the menu to export the process or threads table data to CSV files. These can be easily imported into other applications like for example excel or LibreOffice Calc. The below table is the result of a CSV import and adding some calulated fields (the percent ‘of cpu usage’ is calculated by UserTime/Duration*100).
There is another export which first transforms the data and then exports to a CSV file. This CSV file is much easier to handle for analysis. It contains the recorded times, all processes and the time they spend in user (cpu usage) mode. The export does something like rotating the ‘processes’ table.
#region Transform
class PROCESS_USAGE
{
public string procname;
public int user;
public UInt64 timestamp;
public PROCESS_USAGE(string sProcessName, int iUserTime, UInt64 iTimeStamp)
{
procname = sProcessName;
user = iUserTime;
timestamp = iTimeStamp;
}
}
public int export2CSV2(string sFileCSV)
{
//### setup
sql_cmd = new SQLiteCommand();
sql_con = new SQLiteConnection();
SQLiteDataReader sql_rdr;
connectDB();
if (sql_con.State != ConnectionState.Open)
{
sql_con.Close();
sql_con.Open();
}
sql_cmd = sql_con.CreateCommand();
long lCnt = 0;
//### Build a List of known processes
sql_cmd.CommandText = "Select DISTINCT Process from processes order by Process";
List<string> lProcesses= new List<string>();
sql_rdr = sql_cmd.ExecuteReader();
while (sql_rdr.Read())
{
lProcesses.Add((string)sql_rdr["Process"]);
}
sql_rdr.Close();
sql_rdr.Dispose();
//create a new table with the process names as fields
string sProcField = "";
foreach (string sProc in lProcesses)
{
sProcField += "[" + sProc + "] INTEGER,";
}
sProcField = sProcField.TrimEnd(new char[] { ',' });
sProcField = "[Time] INTEGER, " + sProcField;
//delete existing table
lCnt = executeNonQuery("DROP Table IF EXISTS [ProcUsage] ;");
//create new one
lCnt = executeNonQuery("Create Table [ProcUsage] (" + sProcField + ");");
//### get all process,user,time data
List<PROCESS_USAGE> lProcessUsages = new List<PROCESS_USAGE>();
sql_cmd.CommandText = "Select Process,User,Time from processes order by Time";
sql_rdr = sql_cmd.ExecuteReader();
while (sql_rdr.Read())
{
string sP = (string)sql_rdr["Process"];
int iUT = Convert.ToInt32(sql_rdr["User"]);
ulong uTI = Convert.ToUInt64(sql_rdr["Time"]);
lProcessUsages.Add(new PROCESS_USAGE(sP, iUT, uTI));
}
sql_rdr.Close();
//### get all distinct times
List<ulong> lTimes = new List<ulong>();
sql_cmd.CommandText = "Select DISTINCT Time from processes order by Time";
sql_rdr = sql_cmd.ExecuteReader();
while (sql_rdr.Read())
{
lTimes.Add(Convert.ToUInt64(sql_rdr["Time"]));
}
sql_rdr.Close();
string sUpdateCommand = "";
//### file the new ProcUsage table
SQLiteTransaction tr = sql_con.BeginTransaction();
foreach (ulong uTime in lTimes)
{
System.Diagnostics.Debug.WriteLine("Updating for Time=" + uTime.ToString());
//insert an empty row
sql_cmd.CommandText = "Insert Into ProcUsage (Time) VALUES(" + uTime.ToString() + ");";
lCnt = sql_cmd.ExecuteNonQuery();
foreach (string sPro in lProcesses)
{
//is there already a line?
// http://stackoverflow.com/questions/4495698/c-sharp-using-listt-find-with-custom-objects
PROCESS_USAGE pu = lProcessUsages.Find(x => x.procname == sPro && x.timestamp == uTime);
if (pu != null)
{
System.Diagnostics.Debug.WriteLine("\tUpdating User="+ pu.user +" for Process=" + sPro);
//update values
sUpdateCommand = "Update [ProcUsage] SET " +
"[" + sPro + "]=" + pu.user +
" WHERE Time=" + uTime.ToString() + //" AND Process=" + "'" + sPro + "'"+
";";
sql_cmd.CommandText = sUpdateCommand;
lCnt = sql_cmd.ExecuteNonQuery();
}
}
}
tr.Commit();
lCnt = 0;
SQLiteDataReader rdr = null;
System.IO.StreamWriter sw = null;
try
{
sw = new System.IO.StreamWriter(sFileCSV);
string sFields = "";
List<string> lFields = new List<string>();
lFields.Add("Time");
lFields.AddRange(lProcesses);
foreach (string ft in lFields)
{
sFields += ("'" + ft + "'" + ";");
}
sFields.TrimEnd(new char[] { ';' });
sw.Write(sFields + "\r\n");
sql_cmd.CommandText = "Select * from ProcUsage;";
rdr = sql_cmd.ExecuteReader(CommandBehavior.CloseConnection);
while (rdr.Read())
{
lCnt++;
sFields = "";
//Console.WriteLine(rdr["ProcID"] + " " + rdr["User"]);
foreach (string ft in lFields)
{
sFields += rdr[ft] + ";";
}
sFields.TrimEnd(new char[] { ';' });
sw.Write(sFields + "\r\n");
sw.Flush();
}
}
catch (Exception) { }
finally
{
sw.Close();
rdr.Close();
}
return 0;
}
#endregion
That is the first time that I used a List<>.Find (PROCESS_USAGE pu = lProcessUsages.Find(x => x.procname == sPro && x.timestamp == uTime);). The ‘rotated’ table looks like this:
Using the above data enables you to create very informative 3D bar charts:
In the chart above you can see the time running from left to right. Each process has its own bar row. The height of the bar shows the relative cpu usage against the measure interval duration of 3000ms (3 seconds).
Possible extensions and planned enhancements
The code is by far not perfect. If you look closer to the exported transformed data, you will recognize empty user time fields. That happens when the receiver thread is not active for example during a time expensive GUI update. So, there are possible improvements.
- Improve decoupling of receive and display of data
- Option to log cpu usage locally on the device (if not to time expensive)
- Integrate better graphics using mschart?
Source Code
Source code at code.google.com
Downloads
Binaries are available inside the google code dirs for cpumon2 and cpumonRcv. For the PC you need the SQLite .Net runtimes installed.
http://www.led-mg.de