A crawler pulled 145.5 GB from 3D-model sites in 34 hours, including 54.8 GB of adult content

What the logs show

In its post of October 26, Open3DLab says a crawler visited nearly all 18,926 projects across its three sites in 34 hours, from a single Amazon Web Services address, and downloaded 1,027 files from 712 projects: 145.5 GB. SmutBase, an adult-only site, supplied 364 files (54.8 GB), and 345 of the 712 projects (62.6 GB) are adult content. 423 of the projects are licensed non-commercial only.

The site blocked the crawler on October 18 after it paused for the weekend; it came back on the 19th and 20th. Two emails to Roblox went unanswered. The post is explicit that it has not verified the user agent's claimed affiliation, and that the idea the data fed Roblox's 3D AI model work is its own speculation based on public research and the wording.

"How can you start an outcry like this?"

One commenter asked how a site can complain while hosting material tagged with a permissive license that includes adult models, saying he could not understand the moral high ground. A reply said the post anticipates this:

"We don't like copyright as it exists. We'd like to see it radically reformed, and we wouldn't mourn it if it were abolished."

The same passage notes many models are fan ports of commercial characters shared for free, and "When a studio sends us a takedown notice, we comply."

Blocking the cloud, and why it stops working

One commenter suggested simply refusing traffic from AWS, DigitalOcean and Azure. Another said another thread pointed out this loses offices that connect through VPN gateways on those clouds. A third answered that residential proxy traffic is already a regular topic on the front page, so crawlers will not stay in datacenters.

If the Roblox link holds, the licensing numbers matter more than the bytes: 423 non-commercial projects were taken by something carrying a corporate contact address.