@mangeurdenuage If they were planning to put it on Steam they might hit a little snag there.
But since I've also be working on the problem of computer-generated Stable Diffusion prompts, I am interested in looking at some of their ideas though.
With SD I settled on using 1 2D drawing model, 1 3D render model, 1 realistic model and 1 creative/versatile model. The last one is one that regardless of quality of style, had the highest probability of giving me what I wanted for even with a difficult prompt.
@Moon@Christmas_Man@meso This is the guy making 4bit quantized models for home use: https://huggingface.co/TheBloke GPTQ models are for GPU based inference, GGML are for CPU based inference (though you can get speed boost from moving some of the load on your GPU).
With 24Gb VRAM, you can run GPTQ 13b to 20b models with room to spare for extended (over 2048 token) context and keeping Stable Diffusion loaded at the same time. Or you are supposed to be just about able to run 30b models with 2048 context on a headless linux machine. Expect double digit tokens per second. Answers will pop up in seconds.
With GGML models your RAM is going to be your limit, and speed is going to depend on CPU, GPU, RAM speed and how much you can offload to GPU/VRAM. But in general it's likely to be MUCH slower than GPTQ, but if you're running as big a model as you can fit in your machine, expect single digit tokens per second. Expect to wait sometimes over a minute for an answer. Sometimes it's worth it, sometimes not. I've heard people say that the returns from 30b to 70b are quite a bit diminished (ie: it's not really noticeably smarter, just different).
@piggo@lain I run it with same number of steps as original pass (set to 0 steps), and the choice of upscaler matters. Latent isn't good. I like 4x Valar and I know a lot of people use 4x UltraSharp.
@lain@coin Yep, we're only just starting to see some finetunes getting merged for SDXL. The best 1.5 checkpoints by now are a convoluted tree of finetunes and merges, so we're still in the very early stages of building that for SDXL. https://civitai.com/models/124419?modelVersionId=136753
As of now, I find the most interesting thing to do is check how SDXL fares with things that SD 1.5 had a hard time doing. Like raccoons :blobraccoon: