Laravel job gets picked up multiple workers even though TTR is set to high values

I am trying to work on a job which basically hits third party APIs and gets back data to insert it into the database. Sometimes this data takes more than a while to come and insert.

I am using beanstalkd as my queue driver and as I understand that if the job takes more than 60 secs, which is the default TTR, it will be pushed back into the ready queue waiting to be picked up by another workers even if it is still being processed by a worker who picked it up in the first place. So as a result, 2 workers pick up the same job and I sometimes end up with doublets or even triplets of the same database entries.

So I did the most logically thing that appeared to me that is to increase the TTR to 6000 which is close to 2 hours and my jobs can't possible take that long. It takes 3-5 mins for a job to process.

But even though I set it to such a high value, somehow multiple jobs are still picking up the job. What am I missing? There seems to be some kind of race condition which I am not able to solve. Any ideas anyone?

from Newest questions tagged laravel-5 - Stack Overflow http://ift.tt/2ijv8NA
via IFTTT