App: Recognize
The recognize app provides media tagging and face recognition functionality for the photos app. Recognize can group similar faces on user’s photos (“face recognition”); it can add fitting tags to photos detecting landscapes, food, vehicles, buildings animals and other objects, as well as known landmarks and monuments; it can recognize music genres in user’s audio files and adds tags for those; it can recognize human actions on user’s video files and add tags for them. It specifically runs only open source models and does so entirely on-premises. Nextcloud can provide customer support upon request, please talk to your account manager for the possibilities.
Front-end
Tagged files will appear in the Photos app under the “Tags” section as well as in the normal Files app. Face recognition results will appear under the “People” section in the Photos app.
Requirements
Nextcloud AIO is not supported but will likely work at sub optimal speed
Minimum supported Nextcloud version: 26
x86 CPU
GNU lib C
Background Jobs must be executed via cron
Using GPU processing is supported, but not required; slow performance is expected if you are not using a GPU
We currently only support NVIDIA GPUs
For GPU support you need to install:
NVIDIA® GPU drivers version 450.80.02 or higher.
CUDA® Toolkit 11.x
cuDNN SDK 8.x
GPU Sizing
The models used by recognize require about 1GB of VRAM or less
CPU Sizing
If you don’t have a GPU, this app will utilize your CPU cores
The more cores you have and the more powerful the CPU the better, we recommend 10-20 cores
In the app settings you can set the number of cores to use
At least ~4GB of RAM dedicated for recognize
Disk space usage
~1.5GB for all models in total
Installation
Install the recognize app via the “Apps” page in Nextcloud, or by executing
occ app:enable recognize
Execute the following command on your server terminal of each node that runs background jobs:
occ recognize:download-models
Go to your Nextcloud Administration settings and open the recognize admin settings page
Enable all modes of operation that you want the app to undertake
Enable GPU mode if you have a GPU that you want to use; if you want to use CPU only, you can set the number of cores to use here
Execute the following command on your server terminal to stop background processing of existing files:
occ recognize:clear-background-jobs
Execute the following command on your server terminal to process all existing files in bulk (This may take a long time, depending on how many files you have on your instance):
occ recognize:classify
Execute the following command on your server terminal to calculate face clusters from faces found in all existing files (Run this repeatedly until no more clusters are found):
occ recognize:cluster-faces
All new files from this point on will be automatically processed in background tasks without manual intervention
Scaling
It is possible to scale this app by adding multiple “background” nodes to your cluster that will only process background jobs by executing cron.php.
App store
You can also find the app in our app store, where you can write a review: https://apps.nextcloud.com/apps/recognize
Repository
You can find the app’s source repository on GitHub where you can report bugs and contribute fixes and features: https://github.com/nextcloud/recognize
Nextcloud customers should file bugs directly with our Support system.
Known Limitations
Make sure to test whether the functionality meets the use-case’s quality requirements
Machine learning models notoriously have a high energy consumption
Customer support is available upon request, however we can’t solve false or problematic output, most performance issues, or other problems caused by the underlying model. Support is thus limited only to bugs directly caused by the implementation of the app (connectors, API, front-end, AppAPI)
Ethical AI Rating
Rating for Photo object detection: Green
Positive:
The software for training and inference of this model is open source
The trained model is freely available, and thus can be run on-premises
The training data is freely available, making it possible to check or correct for bias or optimize the performance and CO2 usage.
Rating for Photo face recognition: Green
Positive:
The software for training and inference of this model is open source
The trained model is freely available, and thus can be run on-premises
The training data is freely available, making it possible to check or correct for bias or optimize the performance and CO2 usage.
Rating for Video action recognition: Green
Positive:
The software for training and inferencing of this model is open source
The trained model is freely available, and thus can be ran on-premises
The training data is freely available, making it possible to check or correct for bias or optimize the performance and CO2 usage.
Rating Music genre recognition: Yellow
Positive:
The software for training and inference of this model is open source
The trained model is freely available, and thus can be run on-premises
Negative:
The training data is not freely available, limiting the ability of external parties to check and correct for bias or optimise the model’s performance and CO2 usage.
Learn more about the Nextcloud Ethical AI Rating in our blog.