mardi 11 février 2020

Architecture concept for a background synchronize application

I'm currently struggling with the architecture of my application while refactoring it. It is supposed to run most of the time in background driven by cronjobs as well as queued jobs managed by 'supervisor'.

The purpose of the app is to gather document-data from an API (source system) and distribute this data to different destinations (APIs, filesystem, etc.).

My approach is to split it into different modules to become more flexible and offer better extensibility. Each module would have its own services and some of them their own repositories and models. Not each service would have a matching model and/ or repository. Basically I only have one table with the document data. But there are many many checks and cases to make to bring the data into the right structure for the respective target/ distribution. So I would split the logic into multiple services per module. For example, one service is responsible to process the metadata while another one downloads the file relating to the document while the third one receives data from another API e.g. for translation.

The first module is only focused on the source system, retrieving changed documents. For that, it can run a crawler (especially to build an initial index) or ask the source system directly for changes in a certain amount of time in the past. It is also responsible for updating the index (database of all documents). Because of the data from the source needed in the respective type of distribution can differ, I would prefer to always store all available data.

This is the first point where I'm a bit uncertain because of the SingleResponsibilityPrinciple. Should I split the first module into two single ones? One only responsible for receiving the data and another one to store it to the index? Or is it ok, to gather and store at the same time since there is only one source system and therefor only one index?

Other modules distribution modules) would have business logic to transform the data from the index into the desired structure, so only they know about these specialties.

While some changes at the documents should be distributed as soon as possible, other distribution modules should do their work only once per day or just manually triggered. At the same time, I want to provide all functionalities as manually triggered by a frontend. This is to patch an error that happens during the automatic run (e.g. just 'redo' anything) or to test some processing. Also, especially the tests have to run with special parameters respectively under different environmental conditions. While this could be pretty similar to unit tests, these tests are more focused on the processing of the 'real' data since some errors could occur while processing which than could be fixed also in the source system.

How can I provide functionality as a scheduled command as well as from a controller method while doing the latter, it should be possible to change it's behavior a bit. For example, if a changed document is recognized by the 'automated part' of the app, a job is dispatched to request an API with these changes. I also want to run the logic of the respective distribution module manually triggered for any (not necessarily changed) record of a document without having a job and it should not be sent to its destination but the created request should be printed to the user in the web-frontend.

When I think from a common perspective with a frontend, I would send a request to a controller method. There I would inject multiple services (which can turn also depend on other service classes) and use them within the controller method.

Is it a good practice to then call the controller method from the scheduled command since the frontend is not the real core of the application but it's the command doing its job without any human interaction commonly. Or should I create a "main"-service for each module which then depends on other service-classes while any point of the app which needs this functionality, just calls this "main"-service?

In my current state of knowledge, I always need a place where I call different methods in a certain order. At this place, no checks or business logic should be done. Basically this could be a controller method. But I also already know, that I have to keep a controller as small as possible.

I hope I could make this a bit understandable and I would be very glad if I could get some advice, how to build this kind of modular application.

Thanks a lot, Danaq



from Newest questions tagged laravel-5 - Stack Overflow https://ift.tt/2SzRglD
via IFTTT

Aucun commentaire:

Enregistrer un commentaire