so every single repository should have to spend their time, energy, and resources on accommodating a bunch of venture funded companies that want to get all of this shit for free without contributing to these repositories at all themselves? You think that is a fair ask? That these (often underfunded) institutions should have to accommodate the American private sector’s free lunch because they’re entitled to break our sites without warning?
Honestly the more I write the more this sounds like capitulating to hackers.
so every single repository should have to spend their time, energy, and resources on accommodating a bunch of venture funded companies that want to get all of this shit for free without contributing to these repositories at all themselves?
Was Aaron Schwartz wrong to scrape those repositories? He shouldn’t have been accessing all those publicly-funded academic works? Making it easier for him to access that stuff would have been “capitulating to hackers?”
I think the problem here is that you don’t actually believe that information should be free. You want to decide who and what gets to use that “publicly-funded academic work”, and you have decided that some particular uses are allowable and others are not. Who made you that gatekeeper, though?
I think it’s reasonable that information that’s freely posted for public viewing should be freely viewable. As in anyone can view it. If they want to view all of it and that puts a load on the servers providing it, but there’s an alternate way of providing it that doesn’t put that load on the servers, what’s wrong with doing that? It solves everyones’ problems.
Okay, look, the reason people are disagreeing with you is that you’re responding to the following problem:
“Private companies are preventing access to public resources due to their rapacious, selfish greed.”
And your response has been:
“By changing how we structure things to make it easier for them to take things, we can both enjoy the benefits of the public resources.”
The companies are not the same as normal patrons. They’re motived by a desire for infinite growth and will consume anything that they can access for low prices to resell for high ones. They do not contribute to these public resources, because they only wish to abuse them for the potential capital they have.
Drawing an equivalence between these two things requires the willful disregard of this distinction so that you can act as if the underlying moral principle is being betrayed because your rhetorical opponent didn’t define it as rigorously as possible. They didn’t do that out of an expectation that you would engage with this in good faith.
Yes, I know the companies are not the same as normal patrons. I don’t care that they’re not the same as normal patrons. All I’m concerned about is that the normal patrons get access to the data. The solution I proposed does that.
The problem, as I see it, is that’s not all that you are concerned about. Your goal also includes a second aspect; you want those companies to not have access to that data. So my proposal is not acceptable because it doesn’t thwart those companies.
I’m not drawing an equivalence between companies and individual patrons, I’m just saying my goals don’t include actively obstructing those companies. If they can get what they want without interfering with what the normal patrons want, why is that a bad thing?
I don’t think you understand what is mechanically occurring. He was not putting strain on public servers, he downloaded on-site as one person at a reasonable rate, and then distributed it to the public. It was essentially ethical piracy. No site or entity was put under strain. No one was denied access.
The reason I drew the comparison is his treatment as one person downloading journals and releasing them, vs AI companies scraping countless website/public repositories, taking them down for the public in the process, and then monetizing it internally.
The reason they are being compared is their treatment for extracting publicly funded information. AI companies are being far more destructive. It’s not even close. They are actively harming public data and access with their unfettered sense of entitlement.
If someone did an Aaron-Schwartz-style scrape, then published the data they scraped in a downloadable archive so that AI trainers could download it and use it, would you find that objectionable?
Certainly far less objectionable than taking down public resources, though there’s more to it than that - again, it puts the onus on everyone else to protect themselves from companies that are essentially acting like malicious hackers, Companies that should be the ones responsible for not tearing down public resources. But I don’t really get what you’re trying to prove, because your proposal is not what they’re doing. They’re just doing whatever the fuck they want and don’t care who it impacts. They never do.
I don’t feel like this is very complicated. I’m not allowed to block public roads with my car. I’m not allowed to cut the power to a library and bar the doors. You can’t just deny people public resources like that as a private entity, unless of course you are an AI slop company, in which case states literally aren’t even allowed to make rules about you for the next decade due to our corrupt commander-in-chief. These AI companies are allowed to steamroll any private or public entity they want so long as they condense the right people they will make them a lot of money. It is wildly unethical and the fact that I have to spend so much time convincing you they deserve a little more scrutiny is kind of baffling.
Aaron Schwartz didn’t do anything like the above and your insistence that he is somehow critical to proving some perceived hypocrisy or inconsistency on my part is…well, i’m not sure what the word is, but it’s just not accurate at all.
That suggestion is exactly the same as what I started with when I said “IMO the ideal solution would be the one Wikimedia uses, which is to make the information available in an easily-downloadable archive file.” It just cuts out the Aaron-Schwarts-style external middleman, so it’s easier and more efficient to create the downloadable data.
I have said it twice already, but I will do it a third time:
It is not right to expect everyone else to accommodate private, venture capital fueled AI companies. This is their problem, they are the ones who have to train their models, so they are the ones who have to figure out how to get the data without fucking everyone else in the process. They are not entitled to breaking everything and going “whoopsies!”
I don’t understand why the burden is on the victims here. You are telling libraries et al that it is their responsibility to keep corporations from breaking into their homes, scattering everything everywhere, and forcing them to clean it up themselves.
I don’t understand why the burden is on the victims here.
They put the website up. Load balancing, rate limiting, and such go with the turf. It’s their responsibility to make the site easy to use and hard to break. Putting up an archive of the content that the scrapers want is an easy and straightforward thing to do to accomplish this goal.
I think what’s really going on here is that your concern isn’t about ensuring that the site is up, and it’s certainly not about ensuring that the data it’s providing is readily available. It’s that there are these specific companies you don’t like and you just want to forbid them from accessing otherwise freely accessible data.
That is absolutely ridiculous. The pressure AI scraping puts on sites vastly outstrips anything people built for, as evidenced by the fact that the systems are going down. Building out for that kind of assault costs a lot of money and time. I’m honestly wondering if you understand what it takes.
You know these systems are underfunded. You know these people are underpaid and underappreciated. This is absolutely ass backwards and I don’t understand why you’re defending these companies that are getting unholy amounts of money to inflict this upon our public resources. Do you know what happens if I do this to websites? It’s called a DDoS attack and I get a visit from the feds.
And for the record, I am all about the data being readily available. Readily available to all of us. I don’t care if AI companies use the data or acquire it, but if it comes at the cost of our access, then yes I am opposed. You should be too! Yet here you are misrepresenting the situation and drawing Incredibly crooked parallels.
AI evangelists are all the same, they can’t see beyond the religion they’ve built. Anyone that remotely questions the process these companies feel entitled to are branded as Luddites and shouted down. This can’t be emphasized enough. Do not buy into their cult.
Marketing firms have to pay for surveys or data acquired by other organizations. Political analysts have to pay for polling data or pay to conduct their own.
Let me ask you this: Why are AI companies special? Why do they get to take out my public library without warning and profit off of the act? You keep trying to reestablish what I am saying, so how about you actually express what you believe.
so every single repository should have to spend their time, energy, and resources on accommodating a bunch of venture funded companies that want to get all of this shit for free without contributing to these repositories at all themselves? You think that is a fair ask? That these (often underfunded) institutions should have to accommodate the American private sector’s free lunch because they’re entitled to break our sites without warning?
Honestly the more I write the more this sounds like capitulating to hackers.
Was Aaron Schwartz wrong to scrape those repositories? He shouldn’t have been accessing all those publicly-funded academic works? Making it easier for him to access that stuff would have been “capitulating to hackers?”
I think the problem here is that you don’t actually believe that information should be free. You want to decide who and what gets to use that “publicly-funded academic work”, and you have decided that some particular uses are allowable and others are not. Who made you that gatekeeper, though?
I think it’s reasonable that information that’s freely posted for public viewing should be freely viewable. As in anyone can view it. If they want to view all of it and that puts a load on the servers providing it, but there’s an alternate way of providing it that doesn’t put that load on the servers, what’s wrong with doing that? It solves everyones’ problems.
Really?
Okay, look, the reason people are disagreeing with you is that you’re responding to the following problem:
“Private companies are preventing access to public resources due to their rapacious, selfish greed.”
And your response has been:
“By changing how we structure things to make it easier for them to take things, we can both enjoy the benefits of the public resources.”
The companies are not the same as normal patrons. They’re motived by a desire for infinite growth and will consume anything that they can access for low prices to resell for high ones. They do not contribute to these public resources, because they only wish to abuse them for the potential capital they have.
Drawing an equivalence between these two things requires the willful disregard of this distinction so that you can act as if the underlying moral principle is being betrayed because your rhetorical opponent didn’t define it as rigorously as possible. They didn’t do that out of an expectation that you would engage with this in good faith.
Why are you doing this?
Yes, I know the companies are not the same as normal patrons. I don’t care that they’re not the same as normal patrons. All I’m concerned about is that the normal patrons get access to the data. The solution I proposed does that.
The problem, as I see it, is that’s not all that you are concerned about. Your goal also includes a second aspect; you want those companies to not have access to that data. So my proposal is not acceptable because it doesn’t thwart those companies.
I’m not drawing an equivalence between companies and individual patrons, I’m just saying my goals don’t include actively obstructing those companies. If they can get what they want without interfering with what the normal patrons want, why is that a bad thing?
I don’t think you understand what is mechanically occurring. He was not putting strain on public servers, he downloaded on-site as one person at a reasonable rate, and then distributed it to the public. It was essentially ethical piracy. No site or entity was put under strain. No one was denied access.
The reason I drew the comparison is his treatment as one person downloading journals and releasing them, vs AI companies scraping countless website/public repositories, taking them down for the public in the process, and then monetizing it internally.
The reason they are being compared is their treatment for extracting publicly funded information. AI companies are being far more destructive. It’s not even close. They are actively harming public data and access with their unfettered sense of entitlement.
If someone did an Aaron-Schwartz-style scrape, then published the data they scraped in a downloadable archive so that AI trainers could download it and use it, would you find that objectionable?
Certainly far less objectionable than taking down public resources, though there’s more to it than that - again, it puts the onus on everyone else to protect themselves from companies that are essentially acting like malicious hackers, Companies that should be the ones responsible for not tearing down public resources. But I don’t really get what you’re trying to prove, because your proposal is not what they’re doing. They’re just doing whatever the fuck they want and don’t care who it impacts. They never do.
I don’t feel like this is very complicated. I’m not allowed to block public roads with my car. I’m not allowed to cut the power to a library and bar the doors. You can’t just deny people public resources like that as a private entity, unless of course you are an AI slop company, in which case states literally aren’t even allowed to make rules about you for the next decade due to our corrupt commander-in-chief. These AI companies are allowed to steamroll any private or public entity they want so long as they condense the right people they will make them a lot of money. It is wildly unethical and the fact that I have to spend so much time convincing you they deserve a little more scrutiny is kind of baffling.
Aaron Schwartz didn’t do anything like the above and your insistence that he is somehow critical to proving some perceived hypocrisy or inconsistency on my part is…well, i’m not sure what the word is, but it’s just not accurate at all.
That suggestion is exactly the same as what I started with when I said “IMO the ideal solution would be the one Wikimedia uses, which is to make the information available in an easily-downloadable archive file.” It just cuts out the Aaron-Schwarts-style external middleman, so it’s easier and more efficient to create the downloadable data.
I have said it twice already, but I will do it a third time:
It is not right to expect everyone else to accommodate private, venture capital fueled AI companies. This is their problem, they are the ones who have to train their models, so they are the ones who have to figure out how to get the data without fucking everyone else in the process. They are not entitled to breaking everything and going “whoopsies!”
I don’t understand why the burden is on the victims here. You are telling libraries et al that it is their responsibility to keep corporations from breaking into their homes, scattering everything everywhere, and forcing them to clean it up themselves.
They put the website up. Load balancing, rate limiting, and such go with the turf. It’s their responsibility to make the site easy to use and hard to break. Putting up an archive of the content that the scrapers want is an easy and straightforward thing to do to accomplish this goal.
I think what’s really going on here is that your concern isn’t about ensuring that the site is up, and it’s certainly not about ensuring that the data it’s providing is readily available. It’s that there are these specific companies you don’t like and you just want to forbid them from accessing otherwise freely accessible data.
That is absolutely ridiculous. The pressure AI scraping puts on sites vastly outstrips anything people built for, as evidenced by the fact that the systems are going down. Building out for that kind of assault costs a lot of money and time. I’m honestly wondering if you understand what it takes.
You know these systems are underfunded. You know these people are underpaid and underappreciated. This is absolutely ass backwards and I don’t understand why you’re defending these companies that are getting unholy amounts of money to inflict this upon our public resources. Do you know what happens if I do this to websites? It’s called a DDoS attack and I get a visit from the feds.
And for the record, I am all about the data being readily available. Readily available to all of us. I don’t care if AI companies use the data or acquire it, but if it comes at the cost of our access, then yes I am opposed. You should be too! Yet here you are misrepresenting the situation and drawing Incredibly crooked parallels.
AI evangelists are all the same, they can’t see beyond the religion they’ve built. Anyone that remotely questions the process these companies feel entitled to are branded as Luddites and shouted down. This can’t be emphasized enough. Do not buy into their cult.
Marketing firms have to pay for surveys or data acquired by other organizations. Political analysts have to pay for polling data or pay to conduct their own.
Let me ask you this: Why are AI companies special? Why do they get to take out my public library without warning and profit off of the act? You keep trying to reestablish what I am saying, so how about you actually express what you believe.