Topic: Downloading Favorites in Mass

Posted under General

With E6AI shutting down soon I wanted to save my favorites to my computer, but with 36 pages of favorites that would take awhile to do manually.

Does anyone know of a way to bulk download images? I know there are tools that work for E621, but they don't seem to work for this site.

I've been using Hydrus Network with it's browser extension Hydrus Companion. Works very well, but Hydrus has a fairly steep learning curve.

I always use gallery-dl, since it can handle e621 and FA too

Linux example:

gallery-dl -a browser --cookies-from-browser firefox --destination ./ --write-metadata --write-info-json --download-archive ./archive.sqlite 'https://e6ai.net/posts?tags=fav%3Aexample'

I find that Hydrus' downloader can be a bit slow particularly for larger imports, so I've been using gallery-dl configured to also pull metadata to JSON. Between that and downloading the posts+pools databases from https://e6ai.net/db_exports should effectively be enough to get any needed info for later import into the local Hydrus client.

Also, as it turns out, the search page/endpoint appears to cap out at around 56k posts returned for any given search, and with the final post ID here now being permanently locked at 193896, one could, in theory scrape the whole site in four queries using the

id:x..y

special search operator. Hypothetically, with a decent connection, this could be accomplished in around 48 hours and change without tripping rate limits.

katanablade624 said:
I have not. I will give that a go.

Ok, I have RE6ai installed, but how do I run the downloader? I see the settings for it, but I don't understand how to run it.

DNS

Member

katanablade624 said:
Ok, I have RE6ai installed, but how do I run the downloader? I see the settings for it, but I don't understand how to run it.

Search like normal, then scroll down a ways. In the sidebar you'll see the downloader.

Clicking 'select all visible' selects, as its name implies, all visible posts on your current page. The easiest way is just enabling pageless mode (forgot its real name), scrolling all the way down so the next page loads, and so on until all pages are visible.

Then scroll back up to the top where the downloader is in the sidebar, click all visible, click download.

Be advised that my personal observation is that unless the downloader is in view on your display, it slows down or stops otherwise. You can do something in another browser window, but you can't switch tabs in that same browser window.

Also, bulk content is downloaded in folders up to around 750MB, and the further along you get, the longer the time between individual posts becomes as you're working with larger files.

dns said:
Search like normal, then scroll down a ways. In the sidebar you'll see the downloader.

Clicking 'select all visible' selects, as its name implies, all visible posts on your current page. The easiest way is just enabling pageless mode (forgot its real name), scrolling all the way down so the next page loads, and so on until all pages are visible.

Then scroll back up to the top where the downloader is in the sidebar, click all visible, click download.

Be advised that my personal observation is that unless the downloader is in view on your display, it slows down or stops otherwise. You can do something in another browser window, but you can't switch tabs in that same browser window.

Also, bulk content is downloaded in folders up to around 750MB, and the further along you get, the longer the time between individual posts becomes as you're working with larger files.

I found the button just after I posted my message. Though the button only showed up when I went to my favorites via my profile. It didn't show up when I just clicked "Favorites" in the header.

Yerf seems to have a full copy of the site already, which is pretty great. Keep it in your bookmarks for when this one goes down.