I would suggest before accepting a model, a set of prompts are run against the model to see what it is capable of generating, and pass the output into a model trained to identify output which morally and legally should never be generated. This is to avoid models trained on illegal content, or biased to towards illegal images. The vetting can be tuned and tighten up over time. Note I used the word 'output' rather than image. In order to produce something which could do this (and I don't know how to do this), you would want the model being tested to not output an image but output a result set which represents the image, which can then be passed to a vetting model which is trained on 'output' and not images. The purpose being to avoid the vetting process and vetting model creation from having to directly handle illegal images.
Please authenticate to join the conversation.
Awaiting Dev Review
π‘ Feature Request
Over 2 years ago
tridasha
Get notified by email when there are changes.
Awaiting Dev Review
π‘ Feature Request
Over 2 years ago
tridasha
Get notified by email when there are changes.