Hi, I would like to know if it is possible to bypass HTTP API and directly querying from file system in SDK when the server and client have direct access to underlying data storage directory. This would avoid unnecessary HTTP calls in the system and reduce network load.
In our current system large number of cameras will be directing writing central reduct store instance and multiple other services will be reading processing those data. We think querying directly from underlying file data would be more efficient and reduce the network load.
We would like to know if this feature can implemented?
If yes, what is the general overview on how it can be implemented.
Hi @Sherzod_N, This is an interesting question. It’s not an SDK part, since they are pure HTTP clients. Direct access to the files from there would be too complicated, as ReductStore performs many optimisations to handle file I/O operations as efficiently as possible.
However, we could consider two options.
Read-only replicas can be placed behind a reverse proxy or individually to your services: Disaster Recovery | ReductStore . It will still work via the HTTP protocol, but you can balance the load without impacting the ingestion process.
Another option that I’ve never tried is to get the engine structure and initialise it in read-only mode from the following link: reductstore/reductstore/src/storage/engine.rs at 87dc9c9b297d9141d698c3341cde57619f9ada43 · reductstore/reductstore · GitHub . The disadvantage of this approach is that you can bypass the HTTP, but it’s an internal API which can change dramatically between minor releases. This means you will need to fix it for every release, which is necessary because we can change the internal data formats.
If you will need help with any of the options. Fill free to ask here.