Hi,
I'm now evaluating to use Phalcon in my small project. Basically, I will retrieve XML feed frequently (i.e 5 secs) through HTTP GET. Then, this app will have insert xml data into DB (1MB ~ 5MB each XML). Any suggestion that how can I use phalcon to handle these data nicely.
If you need quite amount of time to process the data, you should consider to create a queue job using beanstalk. Phalcon implements beanstalk client since ver 1.1.0. Check https://docs.phalcon.io/en/latest/reference/queue.html.
Below is the skeleton for the worker tasks for processing your data.
<?php
namespace BI\Console\Tasks;
class WorkerBase extends TaskBase
{
protected $_tube = 'default';
protected $_name = null;
protected function process($data,&$error){
//process your xml data here
var_dump($data);
}
public function mainAction($arg)
{
$q = $this->BeanstalkQueue;
while (true) {
$q->watch($this->_tube);
$job = @$q->reserve();
$data = $job->getBody();
$processResult = $this->process($data, $error);
$job->delete();
}
}
}
}
?>As @jeffreycahyono advice, this can be executed on CLI using a queue (such as beanstalkd) or by just simply running a cron job, note that Phalcon can't optimize/improve the I/O bound (network access, filesystem, etc) so retrieving this via web can take several seconds or even minutes making your page look slow.
Check this section (https://docs.phalcon.io/en/1.2.0/reference/whats-next.html#do-blocking-work-in-the-background) out for more alternatives for queuing with PHP.
