Wrote a couple of test [thread enabled] applicatons and noticed one thing, they tend to use alot of CPU cycles [cpu at 90 or more %].
How can i keep my TThread from using up cpu cycles.
Wrote a couple of test [thread enabled] applicatons and noticed one thing, they tend to use alot of CPU cycles [cpu at 90 or more %].
How can i keep my TThread from using up cpu cycles.
This depends on what the TThread is doing in its Execute procedure. Are you performing a 'do while not terminated' loop? Or are you doing something through your Execute routine? A 90% usage in a loop can be solved with the quick and dirty Sleep(x) method. However, depending on your code, this should not be primarily necassary. A 90% usage through normal execution cannot be solved so simply.
It,s s loop [a for loop]. the application does mas inserts into a sql server.
Ok, but are you waiting for data at any point before you start a new batch of inserts? 100% CPU usage in any thread process is usually caused by an infinite wait loop. You would have to force the thread to sleep with a Sleep(1000) or so to tell Windows to focus on something for that time. If you are indeed waiting on data, it is best you create an event dispatcher and handle it with a WaitForSingleObject event handler. Or you could create a Message Queue of your own, and check it every couple of seconds and then do what the message says to do, then go turn over the cpu to some other process or thread; then start again.
A code snippet would be helpful too. But if not here are many discussions on why a 100% CPU usage usually occurs in a thread (infinite loops doing nothing).
Will try and post some code later, i think maybe adding say:
sleep(100);
Application.ProcessMessages;
at the end of each insert might help(just thinking).
but then this will show down the inserting of the data. And as it stands one of the insert we have done had taken 2 days.
maybe i should lookinto finding a faster way to insert data into mssql[Slowest sql server i have worked with] while controling CPU usage with TThreads.
Last edited by Artificial_Intelligence; Jul 22, 2006 at 04:54 PM.
Adding a Sleep and ProcessMessages function after each insert would not cause a significant performance hit. I am thinking this is some real time system. If it is, then your best bet is batch inserts. Construct one large multi-row sql statement instead of doing one row at a time. That way, while you wait on data to reach a specific amount, you can give processing to some other thread or process. As soon as you reach the minimum amount of data, let's say 10 rows, you do the insert. Either way, the only way to stop 100% usage in a seperate thread is to have that thread not eating time waiting on some event.
How would you build that batch insert you speak off? at the moments i use single inserts and transaction at the start and end of the entire insert[beets having to know where the insert stopped]. i have wanted to find a more Efficient insert process for the sql.
I would need to know the nature of the system to best advise that. However here is an example of a plan you would follow if you were capturing data from an IP camera and storing it into an SQL database.
- Determine the type of data you want to capture - Images
- Any conversions or SQL compatible type? - BLOB
- Interval you wish to capture this data - 4 seconds
- Insert plan (Single or Batch) - Batch
- Batch Threshold - 4 consecutive capture events
- Trigger type - Timer
Now the last part is crucial, since we dont want data in real time and only every 4 seconds, we would create a TTimer component and attach its OnTimer event.
In your CameraDataThread in the OnCreate you would set up your variables but make sure you start Suspended! Very important else your thread will run to execution and terminate.Code:IPCameraTimer := TTimer.Create(Self); IPCameraTimer.Interval := 4000; // every 4 secods get data IPCamerTimer.OnTimer := IPCameraTimerEvent; procedure IPCameraTimerEvent(Sender : TObject); var Timer : TTimer; begin Timer := (Sender as TTimer); CameraDataThread.Resume; // resume the thread Timer.Enabled := False; // this is important, since we are using a seperate thread that is asynchronous, we dont want this event firing until the thread is done constructing the insert statement. end;
We will need the following variables:
- FSQLString // TStringList for the SQL Statement
- EventSoFar variable
- Allowable events constant = 4
- Database and Dataset components (should be local to the thread, we do not want to be going back to the main thread to access any components. Most TDataset descendents are thread-safe.
Back to the thread OnCreate event - Create Dataset and FSQLString objects;
In the Execute event you would have something like this:
Now this is a quick and dirty solution. The premise is to keep the thread suspended until we need it to construct our SQL or insert what we have. Please note that accessing the camera component from the execute method is not a good idea unless it is thread-safe, but we assume it is. If it isn't we would have to use a synchronize routine to access it in the main thread. after we have prepared our statement we can execute, clear and suspend the thread again. We use a counter to monitor the events we have. We can get a little more creative incase the counter might be exposed we could create thread-safe counters so that no other thread might modify the EventsSoFar variable while we are using it. This may cause unexpected behaviour. But that is later.Code:while not Terminated do while not Suspended do begin if EventsSoFar < AllowableEvents then begin IPCamera.Capture(Data); //blobs are strings of characters fundamentally FSQLString.Add('INSERT INTO camera_data (datetime, cameradata)'); FSQLString.Add('VALUES (NOW(),' + EscStr(Data) + ');'); // depending on the dataset, it should include a function to convert data into escaped characters for inserting as blobs Inc(EventsSoFar); end else begin Dataset.SQL.Assign(FSQLString); Dataset.ExecSQL; FSQLString.Clear; // clear the sql for the next round of data EventsSoFar := 0; // restart are rude counter end; Suspended := True; // stop the thread execution and wait for the next event end; end;
Now remember, this is QD, that is it is just meant to show how we could approach it. We would need to know what are your specific needs and then work from there.
Happy threading.
Last edited by Blunty Killer; Jul 24, 2006 at 10:45 AM.
Firstly, thanks, the coding sample was good. The application is simply a mas data inserting tool. It takes an "xls" or mdb file and inserts the data into a mssql2000 database->table_based on the data been inserted. On the above matter, I intend to have two connections components as variables of a customized TThread. One for the source data and one for the destination connection.Originally Posted by Blunty Killer
I will pass the connection strings via functions to the variable of the CTThread. The connection components will be created on the fly ,ie myconn=TADOconnection.create();. As you seem to be TThread worthy, ill as you this , how can I create[instantiate] the connection when the thread in created, meaning instead of creating [TADOconnection.create()], components over and over I would create the connections once and have functions pass them the connection strings to use.
Well, ill only do the above if as you say, the TADOconnection and TADOqry are not threadsafe!
Last edited by Artificial_Intelligence; Jul 27, 2006 at 09:49 AM.
Ok good questions. Just going through this has helped me improve a thread system I did some time ago, so I have learned something too.
You're on the right track with the component. Create a TComponent wrapper for the thread. We will have to implement events to communicate between the TThread and the TComponent.
But first, for a persistent FADOConnection then a private FADOConnection in the TThread is one you would create in the Create method and free in your Destroy routine.
When you say you want to communicate between source and destination thread, you want to read from one data source (access or xls), when you have the useful data, write using destination thread to your mssql database. This to be facilitated through the use of a function and variable? I would suggest you look into a thread safe list, one of two routes, tthreadlist or criticalsections. This is because passing data through variables can cause problems if these variables are modified.
Here is a snippet of the TThreadlist from Borland's Help. You would access this list from each thread, each list would hold data structures for your data and you can be sure it will not be modified by another thread when you lock it.Unit
Classes
Syntax
[Delphi] type TThreadList = class (System.TObject);
Description
A TThreadList object is a thread-safe list. Each TThreadList maintains a private TList (a list of pointers to objects). You can add or remove items in a TThreadList from multiple threads without explicit locks.
To access the actual TList object managed by the thread list, first lock the list by calling the LockList method. Unlock the list by calling the Unlock method when done.
Tip:
By default, TThreadList ignores attempts to add duplicate entries to the list. If the list is large, this default is computationally expensive. For better performance, you may want to change the Duplicates property to dupAccept where possible.
So to recap, just use a private FADOConnection variable local to the thread. And to access data between two threads best look to a TThreadList solution.
I have a program I wrote along time ago that made use of threads and databases extensively, I am thinking of putting the source code in the public domain, if you want you can get it and modify it as much as you can, just credit me with the original and you can publish your modifications too. Maybe a Mozilla License. Tell me what you think.