threading question

[Eugene Krel]

I am working on a piece of code and part of it is a producer-consumer queue.

The following is my pop method:

void pop(Data& popped_value)
{
	boost::mutex::scoped_lock lock(m_mutex);

	while (m_queue.empty()) //block while queue is empty
	{
		m_condition_var.wait(lock);
	}

	popped_value = m_queue.front();
	//pop after copying the value to take exceptions into account
	m_queue.pop(); 	
}

The push method notifies.

The calling object destructor calls thread.join() to stop the thread.

The issue is that if the queue is empty it will just sit there waiting for the conditional variable to un-block and won't call the queue's destructor.

Stack trace:

thread 1:
pthread_cond_wait
boost::thread::join()
foo::~foo()

thread2:
pthread_cond_wait
foo::do_stuff_with_queue()

What is the proper way to deal with that?

The naive solution is to create a pop() method that returns immediately if the queue is empty, but that is not multi-producer/consumer safe.

I am not sure that timed_join() is appropriate either since I can't predict how long it would take for the thread to complete if it's actually doing work.

 

[elliot]

I am working on a piece of code and part of it is a producer-consumer queue.

Stack trace:

thread 1:
pthread_cond_wait
boost::thread::join()
foo::~foo()

thread2:
pthread_cond_wait
foo::do_stuff_with_queue()

What is the proper way to deal with that?

.

thread 1 is blocking on join(), so obviously it is waiting for thread 2 to die.
Take a look at ACE http://www.cs.wustl.edu/~schmidt/ACE.html . You will know how to do it.

 

[sampadm]

I will give it a try, though I don't clearly get what you are trying to achieve here also I have not used boost threading much.
Typically if you want to implement multi-threaded push and pop, then the consumer thread has to wait and pop on the queue and the producer thread has push and signal. If you don't want to wait then you can do it serially , why do u have to multithread it. Just pop it after the push.
Or u can wait for a specific time using timed wait, then do sth else and come back to the queue later, u wont be blocking the thread then.
Also, I see few issues in the code:
1/ In ur pop method u should unlock the mutex after u pop it from the queue else it might cause deadlock related issues.
2/ From ur stack trace it looks like both are waiting on the conditional variable. The push method should have pthread_cond_signal to signal the waiting thread.

Not sure if the above makes ur problem any easier or makes sense but good luck to you and do let us know if u figure it out.

 

[Eugene Krel]

The queue is just a thread safe wrapper for std::queue, it's not a consumer or producer by itself. It is used elsewhere and can potentially have data coming in and out from several threads (more than 2).

scoped_lock takes care of unlocking, the beauty of boost

The push method does in fact have a notify method. The issue is that the thread that was pushing may already be killed (remember the only issue I have is on shutdown) and the one popping may be trying to die except it can't since the wrapper pop method is blocking via the conditional var. Yes i could use kill -9, but that means certain things won't get cleaned up.

elliot: I am aware that that is the issue, I think you may have misunderstood what I was asking. I will take a look at ACE a bit later.

 

[pathfinder]

you could check the status of push thread ...before doing the pop and do a forceful notify in thread1 if push has been terminated

void pop(Data& popped_value)
{
boost::mutex::scoped_lock lock(m_mutex);
bool qWasEmpty=false;
while (m_queue.empty()) //block while queue is empty
{
m_condition_var.wait(lock);
qWasEmpty=true;
}
if(/*push thread is killed */ && qWasEmpty )
{
~m_queue(); //call the queue destructor
//handle any exception and return
return;

}
popped_value = m_queue.front();
//pop after copying the value to take exceptions into account
m_queue.pop();
}