Face-To-Many model tested with 20 faces

Summary

The face-to-many pipeline on Replicate is a popular one (10M+ runs to-date) that can turn any face into many different styles. The various inference params can affect the results dramatically: Not setting them right can produce bad results in terms of face id preservation or bad style. Let's explore how to use it properly.

To conduct the test, we used the publicly available Faces dataset in the Magicflow platform. This dataset includes a variety of faces with different age, gender, and ethnicity.

We tested the model against each face and with various inference params. Then we rated the results according to two questions:

Face similarity (how well does the model preserves the face)
Style similarity (how well the style of the new face is)

Grid: Challenge: Clay

Insights

Face Similarity

Style Quality

Decently well at preserving Face Similarity and the style.

The clay style is doing a great job at actually creating faces that look like clay and that are pretty similar to the original face. For best results set denoising_strength=0.65 and instant_id_strength=1

Image Size:

image

Param 1

Preview

denoising_strength=0.4,instant_id_strength=0.5

denoising_strength=0.4,instant_id_strength=1

denoising_strength=0.65,instant_id_strength=0.5

denoising_strength=0.65,instant_id_strength=1

Grid: Challenge: 3D

Insights

Face Similarity

Style Quality

Hard to produce great 3D style while preserving face.

If you want to preserve the face identity you gonna need to pay with just OK 3D style. For best results set denoising_strength=0.65 and instant_id_strength=1

Image Size:

image

Param 1

Preview

denoising_strength=0.4,instant_id_strength=0.5

denoising_strength=0.4,instant_id_strength=1

denoising_strength=0.65,instant_id_strength=0.5

denoising_strength=0.65,instant_id_strength=1

Grid: Challenge: Pixels

Insights

Face Similarity

Style Quality

Works great

The Pixels style can quickly go wrong and just created a bulrry images instead of a pixel art style. To get a nice style you gonna need to compromise on the face id, but its not too bad in this case because of the nature of this style. For best results set denoising_strength=0.65 and instant_id_strength=0.5

Image Size:

image

Param 1

Preview

denoising_strength=0.4,instant_id_strength=0.5

denoising_strength=0.4,instant_id_strength=1

denoising_strength=0.65,instant_id_strength=0.5

denoising_strength=0.65,instant_id_strength=1

Grid: Challenge: Video Game

Insights

Face Similarity

Style Quality

Does the job well

This style actually produces good looking video game assets and preserving the face well. For best results set denoising_strength=0.65 and instant_id_strength=0.5

Image Size:

image

Param 1

Preview

denoising_strength=0.4,instant_id_strength=0.5

denoising_strength=0.4,instant_id_strength=1

denoising_strength=0.65,instant_id_strength=0.5

denoising_strength=0.65,instant_id_strength=1

Grid: Challenge: Emoji

Insights

Face Similarity

Style Quality

Not working great

This style has a room for improvement, the style isn't great and the face identity isn't preserved well too. For best results set denoising_strength=0.65 and instant_id_strength=0.5

Image Size:

image

Param 1

Preview