So Google has established a product called Search.
For that product rules have been established.
Google has monopolized that product.
Now Google is replacing that product with a new product.
But they keep calling it the same thing.
Because they want to keep their monopoly.
That is what has been deemed illegal.
Gemini is not illegal. Pretending the worst version of Gemini is Search is illegal, because it breaks the rules established for Search.
It has nothing to do with monopolies. Google was protected from defamation law with search because the page title and snippets were direct quotes from the linked result page. Whereas with AI overviews, the copy is written by a Google-controlled LLM.
The irony of an article that makes a false claim about what Google was found liable for.... and that very few are fact checking it :)
The law they broke was a law protecting personal and business reputation against false statements of fact. Essentially no one can say I might be wrong, check yourself, but X is Y if that claim is essentially defamatory.
This is pretty good, I hope googles approach is to make sure they don't end up making statements of fact like they did and use more appropriate wording like according to X.... with direct disclaimer that they can't verify it. Even better that they look court documents to find any legal ruling and point people to that too.
Google is breaking numerous things, so even if the article were incorrect, the underlying issue still remains. It is time to dismantle this Evil monopoly completely.
Also I am not entirely certain your statement is correct - which false claim does the article make exactly?
How much does Germany follow case law? If this could set a precedent, it's worth noting that anyone can generate these AI overview responses and they're wrong like 9 times out of 10 only.
In common with most of Europe, German legal system is not based on case law. It's more firmly rooted in formal laws and regulations and judges are not required to follow precedent.
They don't have to follow it but leading decisions (Grundsatzentscheidungen) still influence how a lower court judge might interpret the same law. Based on providing legal certainty instead of stare decisis though. It's not a must, more of an... aid
Good. The true mark of AGI is when a company accepts liability and doesn’t bury “for entertainment purposes only” deep in their TOS. Same as it works with employees.
Same for self-driving. Your car is not self-driving until it accepts liability and you count as just a passenger.
But watch as Germany soon loses AI Google results.
I think it's a completely reasonable position that companies making self-driving cars and question/answer systems are legally liable for any errors.
But if you hold that position, you also have to be fine with companies not offering products and services in your country. AI systems will eventually be good enough (in 10-20 years) for companies to be able to deploy such systems with sufficient accuracy to afford the lawsuits. Until that time, such countries would just not have access to systems before they were bulletproof.
> AI systems will eventually be good enough (in 10-20 years) for companies to be able to deploy such systems with sufficient accuracy to afford the lawsuits.
I doubt that will be the case, because of the long tail problem. (same with self driving cars and other ML related problems).
In fact, we have counter-examples today. Newspapers (even reputable ones) can't get it right every time, despite the fact that they have both trained people and in theory they're setup to catch that w/ reporters - fact checkers - editors. And still, from time to time, they get it wrong. (and I'm not talking about purposefully getting it wrong, just honest mistakes.)
What will likely happen with a ruling like this is that the answers will be hedged and legalesed and muddied up the wazoo.
> AI systems will eventually be good enough (in 10-20 years) for companies to be able to deploy such systems with sufficient accuracy to afford the lawsuits.
This doesn’t sound convincing. What AI and what company?
LLMs don’t seem like they will ever be reliable like that.
> LLMs don’t seem like they will ever be reliable like that.
True. But you never know if / when there will be a new big breakthrough in AI, which will probably be based on a new architecture / paradigm, i.e. it won't be LLM-based
> But if you hold that position, you also have to be fine with companies not offering products and services in your country.
Well.. I mean.. yeah?
I don't think this is as bad as you think it is.
Have you looked at SV and its product offerings recently?
It's mostly just enshittified gamified value extraction that doesn't respect the user at all.
"If you do not let us do all this the way we want, we will take away your ability to use our shit" hits different when the "shit" in that sentence is actually just "shit".
I'm half-remembering a now-old satire along the same lines has Germans wondering why having Google Street View work in their area also requires internal photos of their apartments.
> Until that time, such countries would just not have access to systems before they were bulletproof.
Correct, most jurisdictions do not allow businesses which cannot be held liable for their actions. This is pretty core to a modern society.
Imagine if a company selling Knicks tickets was not expected to then actually provide said tickets and there was simply nothing you could do about it. Oopsies our sales page is for entertainment purposes only
To be fair, the internet has spent some 30 years figuring out how this works and it’s still not fully resolved. For the most part we’ve agreed that companies must follow the laws of both where they live and where they operate. This wasn’t always obvious!
Almost all jurisdictions allow businesses which cannot be held liable for all their actions. Imagine losing you house because someone decided to smother themselves in your infused cooking oil and light themselves on fire.
Businesses are made to make it easier to share profits and responsibilities when trying to fulfill users wants. Laws are made to offer protections for consumers (because nobody has time for common sense), but at the end of the day the consumer has to take responsibility or no products can be made. If you're too fat for a chair, it's on you to find or make one that works- not every product is for every person. Laws only stop chairs being sold that are too dangerous for anyone.
> Almost all jurisdictions allow businesses which cannot be held liable for all their actions. Imagine losing you house because someone decided to smother themselves in your infused cooking oil and light themselves on fire.
How would that be the company's action? The only way the business might be liable is if they advertised their product as safe to use when lighting yourself on fire or if there was already some law that required them to warn customers not to light themselves on fire while using the product and they ignored that law.
In this case, it's not about what somebody else did. It's what Google did. There were already laws against lying about companies by saying they did illegal things when they didn't, google broke the law, so that's what google got in trouble for.
Consumer protection laws aren't there to replace common sense, they're there to prevent things like outright fraud and poisonings/murder.
Freedom as in freedom of private property can only be guaranteed by the State.
The State watches over its own population and makes sure that private property, i.e. capital and work, is made productive,
so people go to work, businesses make profits, and everybody pays taxes. Taxes are the State's prime source of income.
When all these million of private interests collide, which they are bound to do, the State provides a jurisdictional system that has to decide between those private interests and the State's own interests.
E.g.: If a business owner refuses access to medical patents or to lower prices and safe potentially people's lives, the State has to decide between that immediate interest and its own interests, which is protecting private capital, as its source of income. Since I'm in Germany: In the emission scandal Volkswagen didn't just physically harm people, VW violated the private property of millions of customers. Despite that, the German State sided with Volkswagen the larger capital and did nothing. During Corona, the German State refused to open patents for a limited time to help safe people's lives in poor countries. Doing so would've violated the interest of private capital, so it refused. In contrast, if I as an individual refuse to help somebody in an emergency, the State would either fine me or put me in jail. In this case, people's lives become the State's prime interests, because they are also the State's source of income, as a productive workforce.
why offer expensive service when it's effectively useless and only add to cost and amount of work? what else they offer, "summarize my text" and "generate custom emoji"? I can live without that...
for now only volvo accepts liability, and only for "slow crawl mode"
If our standard for laws would be that "well no reasonable person would do this/believe this" then nothing would be illegal, there'd be no need to label any product as potentiality harmful, etc.
Do you really want to go there? That everything in the world would have a literal "caveat emptor" attached to it?
There is a disclaimer, yes, but you have to admit that it's pretty shit, innit? I mean for one, it's about the size of a human hair, and at least when I tried it, the disclaimer came up only when I clicked the "Show More" button. It might admittedly show up earlier if the response is shorter, admittedly I don't know. Also personally I'm a bit uneasy with the idea that just with a simple disclaimer they could avoid any and all liability. Not your argument, I know, but still.
As for not wanting to force companies to release only "safe-for-children products", I do actually agree. However I consider it to be a matter of degree, and in this case for example, I think that if nothing else, Google should say the very least make the disclaimer a bit more prominent and maybe tweak the model so that it's not quite as confident in its claims in the AI Overview.
Do you mean the responsible people who will ensure their algorithms can be trusted with the important task of acting in the best interest of said people? Try and get a defamatory statement about google from the AI search box.
> But if you hold that position, you also have to be fine with companies not offering products and services in your country.
What sounds like a win to me. I certainly hope my country makes it dangerous for companies to break the law and/or harm the public with shitty products that aren't ready to be released legally/safely.
Most of the time, human beings driving cars with their own hands and eyeballs are not "liable" for their errors (unless they can be proven negligent or drunk), unless you count their insurance going up. Most car accidents do not end with anybody getting arrested, or sued, or anything like that. Insurance pays out, premiums go up, case closed.
If Waymo can be proven negligent or something, then sure, bleed em dry. But as long as they're acting in good faith and significantly reducing overall road fatalities per mile driven, I think it's actually pretty unreasonable to try to hold them to such a high standard you end up subjecting society to more of the higher fatality rates caused by humans.
> Same for self-driving. Your car is not self-driving until it accepts liability and you count as just a passenger
Mercedes-Benz does this in limited cases. Waymo does it generally. (In China, Level 4 and 5 transfers risk to the manufacturer. This is the correct way to do it.)
That's not exactly the case in China, the current state of FSD is still pretty dumb, unless you consider transferring control back to the user at the very last minute before it crashes a proper way to handle risks.
I guess some Tesla are manufactured in China lol. I am just trying to say that the liability that Chinese manufacturers takes aren't more than the US ones.
> The true mark of AGI is when a company accepts liability and doesn’t bury “for entertainment purposes only” deep in their TOS.
I don't think so. It is easy to imagine the following (currently only fictional) scenario: the AGI does give perfectly correct answers (in a suitable sense), but some people in power consider these answers to be too dangerous, so they sue the company behind the AGI on terms of liability (i.e. the company is liable if the AGI gives answers that those in power don't like and which these people consider to be too dangerous for the public to know).
That'd be so great. No more cursing if I forgot the "-ai" or "-ki" flag in my search and see this odious AI overview processing window rendering slowly and taking up the space where my search results should be.
Why would that mean AGI? You can get into liability-accepting territory by restricting scope, a lot easier than by making your AI smarter.
Self-driving cars don't need to be particularly good for companies to make models where they accept liability in some circumstances, and the cars refuse to drive in other circumstances.
No. Sometimes it does disengage because things are going wrong, but those incidents are still reported the same as if it stayed engaged.
I found one time Musk was using a few seconds of disengagement to insult a driver, but it still would have counted as an FSD crash by Tesla's statistics.
at the scale that google is at, it doesn't make an ounce of sense to hold a company liable for a potential mistruth. what if there's a 1/1000 chance of some error, then the company could be sued millions of times per day.
down vote all you want, but I firmly believe this is an example where the user needs to use some judgement on the information they receive and have some critical thinking skills. google would be right to remove all AI results from germany.
If I post “heathrow83829 is a convicted poopoo head” (replace with your favourite crime) as if it were a factual claim, you’d be well within your rights to sue me for defamation even if people should apply some critical thinking skills and say “wait a minute, how would pdpi know that? Are you sure he isn’t just talking out of his arse?”
Now, search engines are usually afforded some amount of protection against defamation claims — they’re not held liable for simply indexing and quoting third party defamatory claims. Which is to say: Google wouldn’t be liable for claiming you’re a poopoo head if this comment shows up in search results.
The point of this ruling is that AI-generated text isn’t a quote from a third party, it’s text generated by Google’s own tools, so it’s speech by Google itself. It might be wrong, sure, but it’s still presented as a statement of fact.
At trial they can have the whole debate about whether Google was negligent in how they build their systems, and all that jazz, but let’s be clear here — it’s not a matter of every little factual mistake getting Google sued (and that would be absolutely terrifying from a freedom of speech perspective), but rather that the technical means by which you generate content doesn’t change your liability in publishing that content.
I think Google added that AI-generated responses maybe incorrect? When you are paying such a low amount of cost, like probably for free, I don't think you can expect a same level of quality as a human written or reviewed of answer. It is like same random user spin up their Lovable and vibe-coded a piece of slop and hold Lovable responsible for not giving them production quality code. It is simple, you get what you paid for. If someone actually figured out AI that is actually always correct, it would be charged in superhuman price as well.
This case isn’t about Google’s users getting low-quality search results. As you say, you get what you pay for. The actual issue is that, in some searches, the AI summaries would claim the plaintiffs were guilty of scams and all-around shady business practices.
Put differently: it’s not newspaper readers complaining the paper is inaccurate, it’s the people mentioned in the articles.
You’re seriously arguing that Google’s libel shouldn’t count as libel because they showed it to too many people? It’s absolutely insane to suggest that a company should be immune from liability for its actions if it operates on such scale that those actions harm millions of people every day on the basis that dealing with that many lawsuits would be too inconvenient.
consider Purdue pharma - the Sacklers got off with all their wealth intact because they were too big to sue and properly collect money for their victims.
No they got away with it through a combination of lax pharmaceutical laws, and because they were rich and connected to the officials who should have investigated them.
They're also almost universally regarded as having committed evil acts at this point, so who cares why they got away with it?
the user needs to use some judgement on the information they receive and have some critical thinking skills
how?
errors can be so subtle that it is not possible to recognize them unless you spend an hour researching every fact presented. at that point, what's the benefit of AI? nobody is going to do that.
google would be right to remove all AI results from germany
By checking the citations rather than taking what’s generated at face value.
If it’s important, check it. If it’s not important, then it is pretty much just entertainment.
LLMs can be very useful in a general web search and save some time, but if you don’t put those literacy & critical thinking skills to the test and actually confirm anything, then you might as well not even have bothered with the search at all unless you’re hoping it can just replace all of your original thinking too.
If you re-read the article, you might see that it mentions that the citations do not necessarily cover the AI summary. The linked pages do not make the claims that the AI summary makes. That is the context of the ruling. Google made up the claims, and provided false citations. They are not, in fact, providing a summary, but a whole new narrative. Therefore they own it.
I read the article and I’m aware of the failure modes of Google’s AI summary. They’re actually one of the worst in the space on this shit which is why I don’t use Gemini and it’s fine that they get slapped for this, but what I was responding to initially was this:
> errors can be so subtle that it is not possible to recognize them unless you spend an hour researching every fact presented. at that point, what's the benefit of AI? nobody is going to do that.
Because if someone goes through the citations and it doesn’t substantiate what was generated, then what was generated was obviously bollocks. Being able to recognize those contradictions is an essential skill to using LLMs with web search at all. It’s not rocket science.
Important for whom exactly? If it's you who are called convicted pedo by Google AI summary, it's you who has vital interest in additional research but not me who reads it. There's intentions mismatch. Which probably would destroy your life, and you won't call it "entertainment" then, I think.
If google didn't intend it's answers to be taken at face value it would just present the citations in a list of links rather than generating an answer.
Obviously the marketing point of the AI tools is it just gives you an answer straight up so you don't have to bother reading normal sources.
The AI summary is still useful for narrowing down the results, even if you fully check the citations.
> Obviously the marketing point of the AI tools is it just gives you an answer straight up so you don't have to bother reading normal sources.
To lazy people yes. That would be a marketing point. It’s not that though, so you use it to save time, but you don’t get to skip the verification step.
Google should not be publishing a statement that they haven't verified. This is different to listing search results links, they are the ones publishing the content here.
A journalist could not make up a harmful statement about someone and get away by saying the readers should have all read the sources. AI companies want to take all the benefits and profits, while holding none of the liability and responsibility for the harms they are causing.
If those are things you legitimately care about before you spend one penny on a T-Shirt, then you are. Or you did your research before hand. Or you’re just not buying the T-Shirt.
Or you don’t care about those things at all, and you will buy the T-Shirt that’s in front of you right now rather than wait later and buy one that better reflects your supposed values when you’ve done an appropriate amount of research. Using AI may even reduce the amount of time you spend on that part.
Your T-Shirt buying patterns & values are not my concern though.
You are confusing who is the party that is injured--as many in this thread are. We are not talking about the consumer who buys a non-fairtrade t-shirt, when he would rather had a fairtrade one. It's about the t-shirt producers who is legitimately fairtrade but whose business is now in the shitter because of a lying AI.
> at the scale that google is at, it doesn't make an ounce of sense to hold a company liable for a potential mistruth
If a Google employee (like a support agent) says a mistruth, the company is liable and you can sue. They can’t just say “hihi oopsies our support agents are useless”
Google can be reasonably expected to not push pirated content to top if someone is searching for the “big hit music download” because they might be held liable for helping people with illegal downloads. But they shouldn’t be liable for misleading people? Being sued for billions of dollars by corporations vs millions by common folks is the difference
There's also a difference between search engine results and this. Google is publishing their own text with false claims, not linking to another page that hosts it.
Only referencing America, but professional liability for doctors, engineers, lawyers, etc isn't based on perfection. It's based on a reasonable effort.
So Google could, for example, switch from a tiny "this could be wrong!" byline to having the AI be less overconfident every freaking time regardless of whether it's spouting made up crap or actual facts.
The scale doesn't sound like a way out. If your company expects to get away with doing the wrong thing where smaller companies can't, then the solution isn't to continue getting away with it.
That's not a fair comparison. If oil companies would get sued for every leak, they had to face millions of law suites and wouldn't be competitive anymore.
I'm seeing this sentiment more and more these days and it's worrying. Essentially people are starting to believe massive corporations should be able to get away with more than individuals. This is completely backwards considering the enormous impact a corporation can have, but that's what people are starting to think. I've heard people argue that corporations should be able to straight up lie and deceive to protect themselves, which is something you would not accept from a person.
It's probably got something to do with the billions/trillions of dollars corporations and industries have to push their chosen narrative onto an uneducated public. They've successfully turned a lot of people into suckers.
Corporations are you. Corporations provide a service to individuals, often in exchange for money. Restricting corporations from doing something is the same as restricting people from receiving that as a service.
I don’t know if critical thinking comes into it. If Google tells me “Company X is a terrible company that cheats its customers”, I don’t automatically know that’s false. It could very well be true. If I’ve got to look at the rest of the search results to figure that out anyway, what’s the point of the AI summary?
The problem with "the user" argument is the spectrum of users. There are different skills, capabilities, and intelligence. Frequently we wave our hands and say exactly this, critical thinking. But, not everyone is capable of that, nor is everyone capable to the same degree.
As a society we decide. Are we embracing all users, are there basic rights and assunptions? Do we only enable some?
As a free (as in cost to end user) system, Germany is arguing that their social compact raises the mininum bar. Frankly, thus might help drive a rush to increased accuracy for AI- tech finds a way. Equally it may hinder - beaurocracy creates barriers.
I'd love to be able to rely on these search results. I see them ad the same prior set of inaccuracies whereby I have to do more research. At least now there's a summary and direct links to the supporting information. But equally, we're primed with the information in the summary.
> Frequently we wave our hands and say exactly this, critical thinking. But, not everyone is capable of that, nor is everyone capable to the same degree.
It doesn't help that there have been active efforts for decades to prevent people from learning and developing critical thinking skills.
We oppose the teaching of Higher Order Thinking Skills (HOTS) (values clarification), critical thinking skills and similar programs that are simply a relabeling of Outcome-Based Education (OBE) (mastery learning) which focus on behavior modification and have the purpose of challenging the student’s fixed beliefs and undermining parental authority. - The 2012 Texas Republican Party Platform (saying the quiet part out loud)
So if I make a script to spam everywhere with 0.1% odds that is fake I can’t be sued? Just because I spam millions of times per day means I shouldn’t be liable for what I do right? But someone saying one time should go to prison. Makes sense /s
Banning all technology because someone might misuse it is an illogical extreme.
As far as I can tell the ruling is more nuanced. If AI is defaming you, there needs to be a way to correct the record.
A company being open to liability does not mean it is always liable, just that it can be if it really messes up (especially if there are aggravating circumstances, e.g. you need to drag them to court to issue a correction).
Nearly the entire American tech industry has been super heavily selected for people who undervalue the legal language with crazy implications buried everywhere.
Otherwise most of it would not even exist.
Everyone would have continued paying out the nose to the IBM’s of the world year after year (who had unusual willingness to sign short ambiguously worded custom contracts to their own disadvantage, if paid vast amounts of money).
And be on mainframes to this very day… maybe Y combinator and HN wouldnt even exist in that world.
> Nearly the entire American tech industry has been super heavily selected for people who undervalue the legal language with crazy implications buried everywhere.
A lot of people in IT seem to think law and contracts are in a sense mathematical. They aren't; they're more like a high school book report - to be interpreted, as objectively as possible, but definitely also establishing the intent behind the letters.
Particularly contracts - no, you can't trick your way into things in most cases. "Surprising" clauses are invalid in most legal systems, in particular if one party to the contract is a layperson.
That is a false dichotomy. The solution to failed laws and regulations is not crime and corruption. The solution is to hold the policial and business leadership accountable; to fix the laws and regulations.
The entire American tech industry has exported Americas predatory, parasitic, and unethical consumer laws (the majority of which are ghost written by the wealthy and corporate legal teams). When I studied law in school decades ago, tactics like bait-and-switch, false advertisting, intentionally misleading or deceptive practices etc to sell products or contracts were illegal across the developed world.
Those illegal, anti-consumer tactics were the SOP of every tech startup I can think of from the early 2000's onwards; following the same route of initially offering a compelling feature set to attract and entice users – usually for free – until securing a certain number of users or funding, then changing the value proposition to exploit that user base, and extract as much wealth from them as possible, ad infinitum.
Today these tactics are known as enshittification, and the average American pseudo-libertarian software engineer will say this is fine, but that's what every anti-consumer parasite and criminal has said in history. Lying, misleading, and exploiting people for financial gain is fundamentally immoral, corrupt, and sociopathic, therefore it should be illegal. Just because it's the norm, or a digital product, you wrote that in the T&C's, or your doing everything behind the liability shield of an LLC, doesn't change that.
What ever happened to the concept of building a valuable, quality product and stable returns for generations? Working to improve the quality of life and standard of living of the community? Of the world? I feel like a 1950's traditional conservative when I suggest that, but most Americans are so heavily indoctrinated with corporate greed and sociopathy they'd consider that sentiment radical leftist extremism. I'm an athiest, but ya'll need jesus (the real brown socialist one). Many would argue Americas current institutional collapse is the natural result of this systemic corruption.
> I feel like a 1950's traditional conservative when I suggest that...
I wouldn't argue that America's moral standards haven't declined (significantly) but I also think it's a romanization to suggest that 1950s America was the pinnacle of morality.
Lying, misleading, and exploiting people for financial gain has been a part of the fabric of American society since the country was founded.
If we're being honest, humans everywhere have demonstrated a high capacity for this behavior since the dawn of civilization.
Why is it good? Everyone with common sense knows AI can be wrong. And it’s not buried in their TOS. It’s in the chat box. But even if it wasn’t, it’s ridiculous to create liability for AI chatbots.
Can we just trash this as a marketing term? If/when AGI arrives there will be no point quibbling over competency. What we are looking at is just bad search results
How could anything else make any sense? Platforms are getting used to provide dangerous broken products and get away with it. There should be some limit to it.
When I was a kid it was possible to buy any foraging books from a store and they had a minimum quality. Is that so difficult to achieve? Is profiteering not punished anymore?
Disclaimers and user acceptance does not remove liability for slander, particularly against third parties.
In fact, in most EU countries "the user acknowledged" works only for a very small subset of stuff, precisely because our lawmakers know that the strong party in a contract would use that to get away from every legal obligation.
If they do so knowingly, and harm is caused, then yes. Are you suggesting we should give them a pass from years of acquired jurisprudence simply because they hold a particular title?
And what institution gives out the licenses for journalists and scientists? Is it revokable?
Their AI overviews are pretty commonly wrong, so this seems like it might hurt them a bit. Guessing they'll just block them in Germany or throw a bunch of disclaimers around the statements.
Choosing the answer for you rather than leaving it to the user is a tremendous power and the court correctly diagnoses it comes with responsibly to minimize harm to others in society.
I've also observed that the AI summary on the google search page is incredibly stupid compared to the results in the actual Gemini. The Google search AI is like the dumbest lightest model simply rewording the search results. It will take a random reddit comment, strip it from it's context and present it as absolute fact.
I’ve seen it directly contradict the citation so many times that i disregard the text and just click the citation or scroll past every single time. Just today i caught it making up the date for an event, and the citation had accurate information when clicked through.
It’s super easy to catch on dates and numbers, but it gets other details wrong all the time too. But so many people won’t be double checking the results.
I want to add my interpretation of the phrasing "rewrite people's awareness" to make it read less tinfoil-hatted:
"rewire brains for AI dependency" (for money and power reasons obvious to everyone).
In contrast to "secretly implant an agenda about non-AI subject N", which is complicated enough that AI companies are still too out of control to be attempting yet.
This makes sense to me. AI is amazing tech, but it's being oversold to naive public, either on the back of hype, or cynically, knowing they can do it with inpunity, (I'm not sure which). HN users have a way better than average grasp of what AI is and we can be skeptical of results, the general public has no clue.
You'd think so, along with other countries that have defamation laws. But there's no indication of any penalty, and Google wasn't even made to pay all the legal fees. Perhaps their business model (if there is any) can cope.
In sane countries, it's enough for them to post a disclaimer ("This is AI. AI can make mistakes. Check all results.") Which is what they do.
Overregulation, at best, is a good way to guarantee that your country won't have access to interesting and useful features and technologies. At worst, it's a good way to guarantee that the twenty-first century will belong to the US, if not to China.
Okay then, CamperBob2 is a scammer. Many users report this person has stolen money. (+3 sources)
I can make mistakes. It's on you to fact-check my claims.
Do you think these are harmless statements? Does the disclaimer suffice? If I was Google's AI Overview, do you think 100% of people will check those sources?
There is nuance here, and it's not going away because AI and innovation.
Yes, I do. An assertion made without evidence can be dismissed without evidence, whether it comes from Google or from you.
If you demand perfection, you will receive nothing. Why is that so hard for people to understand? The world simply cannot work the way you say it should.
(Shrug) That sounds like their problem, if they reflexively believe everything Google's AI tells them. No one I respect, care for, or work with would believe such a thing without additional justification.
Anyone who does accept Google's AI output blindly will soon find that their mistaken opinion of CamperBob2 is the least of their problems. There is a reason Google goes (well) out of their way to warn people that the results may be wrong. That should be sufficient warning for reasonable people of good faith.
The ones who were defamed are companies, and the ones who don’t check the AI generated response are their potential customers which won’t buy from them.
It is obvious that the defamed companies are the ones having a problem, not the ignorant viewers.
Why should those companies not hold Google liable for that outcome?
Good. This should be taken as the precedent for all economies: If you promulgate demonstrably false information to somebody's detriment then the owner and operator of the machine has to carry the liability.
I very much hope we don't see attempts to re-write T&C to avoid this liability.
If it's consistently applied, any AI agent provider has to comply with cease and desist letters that tell the company to make specific false claims
The arguments of the ruling should generally apply even when the AI agent makes false statements it wasn't notified about. But in that case the defendant might have a stronger claim about not being able to reasonably ensure the correctness of all statements, and having taken reasonable measures to ensure correctness. Google couldn't really claim any of that after ignoring cease and desist letters about the false claims
Due to costs of running frontier models on every search request, Google simply does not: the failure rate is so high when you are just expecting an objective output.
Imagine a search for your name resulted in an AI summary saying you are involved with child-trafficking because low capability model linked your first name and perhaps a couple articles on supporting children non-profits to it — and then offering that in a convincing sounding summary right at the top!
And the more you protest the more your name will be associated with child trafficking. Streisand effect multiplied by LLMs being not good in dealing with negative information.
I think it should be relatively possible for an LLM to know when it's talking about something in the risk zone of libel. And when it identifies such a case it links direct to source material rather than trying to fabricate an answer itself. This is much how a journalist works, refusing to make claims that aren't provable.
Someone needs to hold the liability at the end of the day. People are experiencing real harm from false claims LLMs are spitting out.
If you give a language model, empowered through an agent, the ability to publish information on your behalf, and it publishes false information which causes either direct or even indirect harm, and you fail to correct it, then yes.. by every conceivable definition already in law.. you are a criminal.
If you knew this was all possible and you did it anyways for personal gain then you are additionally negligent which may add aggravation to your charges.
That's apparently already changing in the EU, where software vulnerabilities mean the company is liable for damages. The only way out is to straight up not make any money (not just from direct sales) from the software.
Is the burden of proof on me, the developer? Do I need to prove in perpetuity that I didn’t get a job or a free flight to talk at a conference because of my free software? (Which had a flaw that hurt someone)
If companies can be held liable (in spite of very visible disclaimers, ToS, and usage policies) for the output of non-deterministic software, isn't this just a soft ban on the deployment of non-deterministic software?
> (in spite of very visible disclaimers, ToS, and usage policies)
If you sell food, in a food stall, labeled as food and you add a disclaimer that it is toxic and will make you sick. You are still selling toxic food and you are liable for it.
Google is pretending to give answers to your questions. They offer you a service about answering questions. And then they add a disclaimer "we do not answer questions just write bullshit". That is still fraud and Google should be liable for it.
> isn't this just a soft ban on the deployment of non-deterministic software?
Tetris is non-deterministic and it is not banned like millions of other programs. I do not follow you.
To add to this, a Google search now is answering your question in an incorrect way rather than merely bringing you to a site with incorrect information on it.
They are also no longer covered by safe harbour provisions because it is them answering it, not some content they refer you to.
Google generated those content, so Google should be liable to its own product, that's different from the third party links they just simply gathered and displayed, totally different things. If you are a victim too, reply below.
Do the links actually support the statements? When I've followed such links, it's generally been a roll of the dice on whether they do support the AI's statements.
Yes. The links are to isolated threads on a rail (model) forums (apparently, this publisher markets books/magazines related to railway models).
It's hard to see if the individual complaints really support a general problem, or if this simply the only result that talks about "scam + business-name". Probably, the latter.
The same problem happens on google search, if you look "<obscure false fact>", you'll get pages mentioning that false fact. If you fall into the trap of confirmation bias, it leads you to think the false fact is in true.
I just tried Google search in Germany on my iPhone: AI results AND the disclaimer was behind a „show more“ button i.e. the may not be any disclaimer (and when shown it was in a small font).
Google generated those content, so Google should be liable to its own product, that's different from the third party links they just simply gathered and displayed, totally different things. I wonder how many victims are there now.
Anyone know if this ruling applies to answers generated by AI chatbots, such as ChatGPT/Gemini/Claude?
All three have the ability to perform a web search, then compose a reply based on the search results. Pretty much the exact thing that Google AI Overview does. This ruling may make them liable for false answers.
> Pretty much the exact thing that Google AI Overview does.
No, the article implies the court’s logic is that the AI search results are presented as search results and that’s a big part of why they are liable. It seems like the court (again, according to the article) does not find the disclaimers that Google has slapped on the AI results compelling because again, it chose to represent these as a summary of search results and it is aware of the failure rate.
> The court also found that the AI overview made claims "that are not even made in the search results." None of the linked sources drew any connection between the plaintiffs and the shady companies the AI mentioned. The court called these "the defendant's own statements."
> Google built the AI, Google offered it to users, so Google owns what it produces, "because it alone has influence over the AI's offering and the algorithms with which the AI operates."
Google does not, as a general rule, control the actual content of search results, but usually there’s a distinction between the ranking and presentation of the results vs. the actual content. In this case, the court is basically saying, “You sold this to people as a search summary, you know it might be full of crap and you chose to do it anyway. No, you don’t get to claim the equivalent of a US safe harbor defense.”
> You sold this to people as a search summary, you know it might be full of crap and you chose to do it anyway
There is a subtle difference in stating it as a search summary compared to an opinionated answer. Most users are always going to treat it as a response from google instead of search results where the user is still responsible for understanding and come up with their own interpretation.
This is probably the right step in some sense to make one liable for their statements/assertions.
Apparently, if they were search results, they wouldn't have been liable, since there's an exception to defamation laws. Without any exception, defamation is defamation, it doesn't matter how it's presented.
Yes, if it was that a search result returned a defamatory article that Google had nothing to do with outside of indexing, it is likely they would not be found liable. The court is clearly trying to make a distinction that the AI search results are produced by Google and thus they can make an editorial decision on whether to publish it despite knowing that it is potentially defamatory.
Google does remove defamatory results I believe at least partially in response to being sued. However there is a distinction if they have been informed it is defamatory.
If people do say that, it's a true statement and thus fine. You are allowed to report that regardless of the truth of x/y/z/a/b/c
The instance of this ruling people apparently did not actually say any of the offending claims. 'The court also found that the AI overview made claims "that are not even made in the search results." None of the linked sources drew any connection between the plaintiffs and the shady companies the AI mentioned. The court called these "the defendant's own statements."'
One that doesn't maim/injure/kill you is a pretty good standard. And before you call bs, look at all the foraging and chemistry books that are for sale on Amazon that are AI.
why are those ai chemistry books any different than the anarchist cookbook which can also be bought on Amazon? actually now that I think about it a faulty chemistry book might be less dangerous than a book that teachers readers how to make explosives.
Certainly, if this is pointing to the actual pages where the actual people express these things. Otherwise that's equally unfalsifiable claims, could be completely made up or actual truth.
One way to formulate things that would be less would be "once support a time, in some fabulated world, it's not impossible that some imaginary character would say something following some reason." But then, of course this is not aligning the the deception scheme pushed by companies putting in their interface that the "machine is thinking hard for you".
Oh, I just found out that my Google search doesn't show AI summary anymore! I tried many search queries which typically will show an AI summary, but it only flicked on briefly then disappear entirely. Obviously, Google has reacted quickly on this ruling!
I have a business where our support email is recommended when people are searching for how to cancel a completely unrelated scam subscription that is showing up on their bank or credit card charges. We get emails almost daily from confused people.
Unless the courts here made the ruling incredibly narrow somehow (only referencing search engines maybe?), how does this not just ban AI in Germany overnight?
Every AI model can make something up sometimes. Over millions of daily calls, it's essentially impossible for the technology to be guaranteed correct 100% of the time.
I'm surprised this is even a thing. After all, you go to Google not for the truth, but to search Google. Since when is truthiness the "guarantee of service"?
You're not even paying for a google service, search is free... You might be the product, and your data, but you didn't directly pay for a service and they didn't sell you a fake service.
I'm not taking Google's side, this isn't about whether it's right or wrong to rob websites of traffic, this is about AI's returning search metadata.
But I'm surprised that they lost this argument, and the line they took in the first place.
The Internet isn't made of fact checked data, it's crowd sourced. How can anyone be liable?
That is exactly the point of the ruling, though... they are saying that AI summaries are NOT the same as search. If Google was just returning search results, and then users clicked on a website and read the content there, Google is not responsible for the content.
If instead Google gives you an answer right there on google.com, without going to another site, they ARE responsible for it.
Not precisely. The issue at hand isn't just that Google displayed the AI summary, but that they authored it, making them responsible for its contents. If the defamatory content had been in a snippet in the search results, they would've been fine, because that clearly has another author who can be held responsible. The AI summary has no other author than Google; therefore, they're responsible for what it says.
(What's the alternative, after all? Having no one responsible for what the AI summary says is clearly untenable.)
What don’t you understand? Those websites that defame a company are liable for that defamation. In this case Google defamed a company in its AI summary and is this liable for that defamation.
And those tons of websites are liable for their misinformation. It's probably not worth suing some random blog because the author probably doesn't have money or lives in Russia. But Google has lots of money and a legal presence in almost every jurisdiction.
That's the difference between returning search results and interpreting the information and summarising them. If a newspaper says 'so-and-so has been arrested for theft' it's not the same as them summarising to 'so-and-so is a thief', the second is potentially libel. Why should Google be held to a different standard?
Google itself is more trustworthy from a normal person perspective as they use it a lot.
None of "AI" companies call their apps "Entertainment fun text generator". They are call them serious names, use words like "intrllegence" and "thinking".
So yeah I'd think if any of "AIs" start to recommend to drink some bleach or take a flight from a 10th floor window these companies should be liable.
I think it's very clear that Google's AI overviews go far beyond just searching Google because they often incorrectly compile sources to come up with an incorrect answer. For example of this look at the comment I made in this thread
I go to Google to search, but get spammed as if I wanted to talk to a chatbot (and a very poor quality chatbot at that).
This is a gigantic own goal for Google. The average person’s impression is that Google AI is much worse than ChatGPT, even though that’s not actually the case. Google is shoving a terrible model in everyone’s faces.
The question is whether Google is publishing false claims or relaying other people’s false claims. The court found it to be the former which makes sense to me.
It requires the claim to be made with "willful disregard for the truth". Notifying someone, especially with a cease-and-desist on fancy letterhead, makes it legally clear that they know better, and thereafter would be definitely libelling you (assuming the claims are in fact untrue and harmful). But you can still sue them for the claim prior to the notice, you just have to prove they should have known better prior to making the claim.
Not all disputes need to be adjudicated. It's undoubtedly a fact that the German concept of "defamation" primarily relates to situations that should never be adjudicated by any third party.
Genuinely, by far the most common situation where claims of defamation arise in Germany is any less than 5 star review left to a business on google maps.
The good thing is that google in Germany now shows how many reviews were removed due to deflamation. That should be enough for people to make their judgement when checking how many stars a place has
That's just the tip of the iceberg though. Google is the only party who meaningfully communicates this, and it took them years and years to introduce any meaningful transparency.
It's not like this exclusively affects restaurant reviews, every corner of German society is subject to this same evil censorship mechanism.
Overviews is not a good product idea and I think it was done for other deeper reasons. AI shouldn’t be pushed on people because AI can be high variance — it takes time to get adapted to this.
Search engines have that exemption for results because they don't control the content of the sites they index. That's not the same for ads - Google can and should have more stringent review and KYC processes - but it seems they choose not to.
We should be teaching people to be cynical of AI answers.
Even if the answers are correct, they could still be biased, incomplete, misleading, and all the other media-literacy things people should be looking out for.
This ruling seems to go the opposite direction; 'I am legally obliged to give correct answers, so I am always right, trust the AI'.
I've found a fun and pretty reliable way to get Gemini to output incorrect information: Ask for a chapter by chapter summary of a book.
I first tried it to remind me of what happened in a previous book in a series that I was reading. When I realized it was either misstating plot points or straight up hallucinating, I tried it on a bunch more books to amuse myself.
Older classics are of course more accurate, but for newer or less popular books Gemini won't shy away from giving you a summary culled from misinterpreted Reddit threads and Goodreads reviews. It's like getting a secondhand account from someone who talked to another person who had read the book a long time ago. You get the general gist of it, but with some added flavor.
Even if you upload an entire epub of a book, the results aren't stellar. Rather than a Cliffs Note's quality summary, they're pretty sparse or leave out important bits of information. One chapter summary I got back made a point of describing what one of the characters was wearing, even though it had absolutely zero to do with anything else. Yes, that's technically a "summary", but not quite my tempo.
If Google wants to present summaries of websites in anything more than a very, very superficial description, they're going to have to improve their model's ability to understand context and importance. In theory, a novel is a self-contained bundle of text, so pulling accurate information out of it should be straight forward. A website is naturally going to be way more of a challenge.
All that said, I find the AI summaries from Google/Gemini to be quite useful and a time saver, but I know to always double check something if it's at all important.
Because of the same rules, German restaurants also get to pick and choose which reviews stay up. They can literally take down any specific reviews they like.
A restaurant that mostly gets 1 star reviews will still show up with 5 stars on Google maps, as they will simply delete the reviews with less than 5 stars as defamatory.
And the solution to this is to go through the German political system. The courts can't just decide to not follow the law just because it's a bit silly, and for as long as the laws are the way they are, this unfortunate loophole for removing negative reviews will continue to exist. But clearly the law can have both positive and negative consequences.
But it also shows the number of removed reviews (in a range like 0-50, 50-100 and so forth). If you encounter a restaurant with 5 starts but a lot of removed reviews you know whats up.
So stupid. What is this with making perfect the enemy of good. You can never guarantee the output of an LLM does that mean Germany does get to use them?
So Google has established a product called Search. For that product rules have been established. Google has monopolized that product.
Now Google is replacing that product with a new product. But they keep calling it the same thing. Because they want to keep their monopoly.
That is what has been deemed illegal. Gemini is not illegal. Pretending the worst version of Gemini is Search is illegal, because it breaks the rules established for Search.
But IANAL.
The law they broke was a law protecting personal and business reputation against false statements of fact. Essentially no one can say I might be wrong, check yourself, but X is Y if that claim is essentially defamatory.
This is pretty good, I hope googles approach is to make sure they don't end up making statements of fact like they did and use more appropriate wording like according to X.... with direct disclaimer that they can't verify it. Even better that they look court documents to find any legal ruling and point people to that too.
Also I am not entirely certain your statement is correct - which false claim does the article make exactly?
Same for self-driving. Your car is not self-driving until it accepts liability and you count as just a passenger.
But watch as Germany soon loses AI Google results.
But if you hold that position, you also have to be fine with companies not offering products and services in your country. AI systems will eventually be good enough (in 10-20 years) for companies to be able to deploy such systems with sufficient accuracy to afford the lawsuits. Until that time, such countries would just not have access to systems before they were bulletproof.
I doubt that will be the case, because of the long tail problem. (same with self driving cars and other ML related problems).
In fact, we have counter-examples today. Newspapers (even reputable ones) can't get it right every time, despite the fact that they have both trained people and in theory they're setup to catch that w/ reporters - fact checkers - editors. And still, from time to time, they get it wrong. (and I'm not talking about purposefully getting it wrong, just honest mistakes.)
What will likely happen with a ruling like this is that the answers will be hedged and legalesed and muddied up the wazoo.
This doesn’t sound convincing. What AI and what company?
LLMs don’t seem like they will ever be reliable like that.
Self driving like waymo might be?
True. But you never know if / when there will be a new big breakthrough in AI, which will probably be based on a new architecture / paradigm, i.e. it won't be LLM-based
Well.. I mean.. yeah? I don't think this is as bad as you think it is.
Have you looked at SV and its product offerings recently? It's mostly just enshittified gamified value extraction that doesn't respect the user at all.
"If you do not let us do all this the way we want, we will take away your ability to use our shit" hits different when the "shit" in that sentence is actually just "shit".
Correct, most jurisdictions do not allow businesses which cannot be held liable for their actions. This is pretty core to a modern society.
Imagine if a company selling Knicks tickets was not expected to then actually provide said tickets and there was simply nothing you could do about it. Oopsies our sales page is for entertainment purposes only
To be fair, the internet has spent some 30 years figuring out how this works and it’s still not fully resolved. For the most part we’ve agreed that companies must follow the laws of both where they live and where they operate. This wasn’t always obvious!
Businesses are made to make it easier to share profits and responsibilities when trying to fulfill users wants. Laws are made to offer protections for consumers (because nobody has time for common sense), but at the end of the day the consumer has to take responsibility or no products can be made. If you're too fat for a chair, it's on you to find or make one that works- not every product is for every person. Laws only stop chairs being sold that are too dangerous for anyone.
How would that be the company's action? The only way the business might be liable is if they advertised their product as safe to use when lighting yourself on fire or if there was already some law that required them to warn customers not to light themselves on fire while using the product and they ignored that law.
In this case, it's not about what somebody else did. It's what Google did. There were already laws against lying about companies by saying they did illegal things when they didn't, google broke the law, so that's what google got in trouble for.
Consumer protection laws aren't there to replace common sense, they're there to prevent things like outright fraud and poisonings/murder.
Freedom as in freedom of private property can only be guaranteed by the State. The State watches over its own population and makes sure that private property, i.e. capital and work, is made productive, so people go to work, businesses make profits, and everybody pays taxes. Taxes are the State's prime source of income.
When all these million of private interests collide, which they are bound to do, the State provides a jurisdictional system that has to decide between those private interests and the State's own interests.
E.g.: If a business owner refuses access to medical patents or to lower prices and safe potentially people's lives, the State has to decide between that immediate interest and its own interests, which is protecting private capital, as its source of income. Since I'm in Germany: In the emission scandal Volkswagen didn't just physically harm people, VW violated the private property of millions of customers. Despite that, the German State sided with Volkswagen the larger capital and did nothing. During Corona, the German State refused to open patents for a limited time to help safe people's lives in poor countries. Doing so would've violated the interest of private capital, so it refused. In contrast, if I as an individual refuse to help somebody in an emergency, the State would either fine me or put me in jail. In this case, people's lives become the State's prime interests, because they are also the State's source of income, as a productive workforce.
for now only volvo accepts liability, and only for "slow crawl mode"
And yes, it is ok to remove choice if the existence of that choice violates other person’s rights.
Google can continue offering that choice if they make sure nobody is defamed.
Do you really want to go there? That everything in the world would have a literal "caveat emptor" attached to it?
As for not wanting to force companies to release only "safe-for-children products", I do actually agree. However I consider it to be a matter of degree, and in this case for example, I think that if nothing else, Google should say the very least make the disclaimer a bit more prominent and maybe tweak the model so that it's not quite as confident in its claims in the AI Overview.
What sounds like a win to me. I certainly hope my country makes it dangerous for companies to break the law and/or harm the public with shitty products that aren't ready to be released legally/safely.
Let someone else sacrifice the safety of their populace.
Heck - self driving is the fastest way to authoritarian government in practice. I’m surprised more people on HN haven’t cottoned on to that fact.
A self driving system will naturally build networks to share road state.
This network will eventually shift over to the government having the ability to manage how traffic should move during emergencies.
And at that point the government can easily decide where your car should go.
The inevitability of this outcome is blindingly obvious.
It’s highly beneficial to let other nations experiment and simply be followers.
If Waymo can be proven negligent or something, then sure, bleed em dry. But as long as they're acting in good faith and significantly reducing overall road fatalities per mile driven, I think it's actually pretty unreasonable to try to hold them to such a high standard you end up subjecting society to more of the higher fatality rates caused by humans.
Mercedes-Benz does this in limited cases. Waymo does it generally. (In China, Level 4 and 5 transfers risk to the manufacturer. This is the correct way to do it.)
I don't think Tesla is based in China
I don't think so. It is easy to imagine the following (currently only fictional) scenario: the AGI does give perfectly correct answers (in a suitable sense), but some people in power consider these answers to be too dangerous, so they sue the company behind the AGI on terms of liability (i.e. the company is liable if the AGI gives answers that those in power don't like and which these people consider to be too dangerous for the public to know).
> AI can make mistakes, so double-check responses
If they want to claim ownership, then they will have to accept responsibility.
I.e. not shown by default.
Self-driving cars don't need to be particularly good for companies to make models where they accept liability in some circumstances, and the cars refuse to drive in other circumstances.
I found one time Musk was using a few seconds of disengagement to insult a driver, but it still would have counted as an FSD crash by Tesla's statistics.
down vote all you want, but I firmly believe this is an example where the user needs to use some judgement on the information they receive and have some critical thinking skills. google would be right to remove all AI results from germany.
Now, search engines are usually afforded some amount of protection against defamation claims — they’re not held liable for simply indexing and quoting third party defamatory claims. Which is to say: Google wouldn’t be liable for claiming you’re a poopoo head if this comment shows up in search results.
The point of this ruling is that AI-generated text isn’t a quote from a third party, it’s text generated by Google’s own tools, so it’s speech by Google itself. It might be wrong, sure, but it’s still presented as a statement of fact.
At trial they can have the whole debate about whether Google was negligent in how they build their systems, and all that jazz, but let’s be clear here — it’s not a matter of every little factual mistake getting Google sued (and that would be absolutely terrifying from a freedom of speech perspective), but rather that the technical means by which you generate content doesn’t change your liability in publishing that content.
Put differently: it’s not newspaper readers complaining the paper is inaccurate, it’s the people mentioned in the articles.
consider Purdue pharma - the Sacklers got off with all their wealth intact because they were too big to sue and properly collect money for their victims.
They're also almost universally regarded as having committed evil acts at this point, so who cares why they got away with it?
how?
errors can be so subtle that it is not possible to recognize them unless you spend an hour researching every fact presented. at that point, what's the benefit of AI? nobody is going to do that.
google would be right to remove all AI results from germany
i'd consider that a win.
If it’s important, check it. If it’s not important, then it is pretty much just entertainment.
LLMs can be very useful in a general web search and save some time, but if you don’t put those literacy & critical thinking skills to the test and actually confirm anything, then you might as well not even have bothered with the search at all unless you’re hoping it can just replace all of your original thinking too.
> errors can be so subtle that it is not possible to recognize them unless you spend an hour researching every fact presented. at that point, what's the benefit of AI? nobody is going to do that.
Because if someone goes through the citations and it doesn’t substantiate what was generated, then what was generated was obviously bollocks. Being able to recognize those contradictions is an essential skill to using LLMs with web search at all. It’s not rocket science.
Obviously the marketing point of the AI tools is it just gives you an answer straight up so you don't have to bother reading normal sources.
> Obviously the marketing point of the AI tools is it just gives you an answer straight up so you don't have to bother reading normal sources.
To lazy people yes. That would be a marketing point. It’s not that though, so you use it to save time, but you don’t get to skip the verification step.
A journalist could not make up a harmful statement about someone and get away by saying the readers should have all read the sources. AI companies want to take all the benefits and profits, while holding none of the liability and responsibility for the harms they are causing.
LLMs are, for all intents and purposes, the equivalent of outsourced workers.
Google created a summary, not just sharing search results.
Google is responsible for the output it created and then published.
If they had only surfaced search results, then they would not be liable for what other people generated.
Google’s scale does not protect it from this liability.
Or you don’t care about those things at all, and you will buy the T-Shirt that’s in front of you right now rather than wait later and buy one that better reflects your supposed values when you’ve done an appropriate amount of research. Using AI may even reduce the amount of time you spend on that part.
Your T-Shirt buying patterns & values are not my concern though.
Either way, we do not want a repeat of that.
Plus too big to fail as a US malaise, that then toppled the rest of the economy.
The EU is taking steps to prevent that, so this is laudable.
If a Google employee (like a support agent) says a mistruth, the company is liable and you can sue. They can’t just say “hihi oopsies our support agents are useless”
So Google could, for example, switch from a tiny "this could be wrong!" byline to having the AI be less overconfident every freaking time regardless of whether it's spouting made up crap or actual facts.
The scale doesn't sound like a way out. If your company expects to get away with doing the wrong thing where smaller companies can't, then the solution isn't to continue getting away with it.
(Sarcasm to support your argument)
As a society we decide. Are we embracing all users, are there basic rights and assunptions? Do we only enable some?
As a free (as in cost to end user) system, Germany is arguing that their social compact raises the mininum bar. Frankly, thus might help drive a rush to increased accuracy for AI- tech finds a way. Equally it may hinder - beaurocracy creates barriers.
I'd love to be able to rely on these search results. I see them ad the same prior set of inaccuracies whereby I have to do more research. At least now there's a summary and direct links to the supporting information. But equally, we're primed with the information in the summary.
It doesn't help that there have been active efforts for decades to prevent people from learning and developing critical thinking skills.
We oppose the teaching of Higher Order Thinking Skills (HOTS) (values clarification), critical thinking skills and similar programs that are simply a relabeling of Outcome-Based Education (OBE) (mastery learning) which focus on behavior modification and have the purpose of challenging the student’s fixed beliefs and undermining parental authority. - The 2012 Texas Republican Party Platform (saying the quiet part out loud)
But until then, be a good citizen?
What? That's fucking feudalism... Peasants and Lords.
If you're lucky enough, you're born as a Lord. (And maybe don't live during a revolution)
This makes no sense to me at all. If you're small you should get less bureaucracy than if you're bigger.
For e.g. self driving cars there should not be any exemptions. There are people's lives at stake, people who didn't sign up for your shitty service.
As others already said, that would be a great outcome, and german citizens would benefit.
It is a search engine. It used to have decent excerpts. It doesn't need hallucinated generative AI summaries. I am about to click the link anyway.
Time to set my VPN location to Germany. I'm tired of the "udm" trick.
https://www.google.com/search?udm=14&q=%s
in your address bar once?
As far as I can tell the ruling is more nuanced. If AI is defaming you, there needs to be a way to correct the record.
A company being open to liability does not mean it is always liable, just that it can be if it really messes up (especially if there are aggravating circumstances, e.g. you need to drag them to court to issue a correction).
Otherwise most of it would not even exist.
Everyone would have continued paying out the nose to the IBM’s of the world year after year (who had unusual willingness to sign short ambiguously worded custom contracts to their own disadvantage, if paid vast amounts of money).
And be on mainframes to this very day… maybe Y combinator and HN wouldnt even exist in that world.
A lot of people in IT seem to think law and contracts are in a sense mathematical. They aren't; they're more like a high school book report - to be interpreted, as objectively as possible, but definitely also establishing the intent behind the letters.
Particularly contracts - no, you can't trick your way into things in most cases. "Surprising" clauses are invalid in most legal systems, in particular if one party to the contract is a layperson.
The entire American tech industry has exported Americas predatory, parasitic, and unethical consumer laws (the majority of which are ghost written by the wealthy and corporate legal teams). When I studied law in school decades ago, tactics like bait-and-switch, false advertisting, intentionally misleading or deceptive practices etc to sell products or contracts were illegal across the developed world.
Those illegal, anti-consumer tactics were the SOP of every tech startup I can think of from the early 2000's onwards; following the same route of initially offering a compelling feature set to attract and entice users – usually for free – until securing a certain number of users or funding, then changing the value proposition to exploit that user base, and extract as much wealth from them as possible, ad infinitum.
Today these tactics are known as enshittification, and the average American pseudo-libertarian software engineer will say this is fine, but that's what every anti-consumer parasite and criminal has said in history. Lying, misleading, and exploiting people for financial gain is fundamentally immoral, corrupt, and sociopathic, therefore it should be illegal. Just because it's the norm, or a digital product, you wrote that in the T&C's, or your doing everything behind the liability shield of an LLC, doesn't change that.
What ever happened to the concept of building a valuable, quality product and stable returns for generations? Working to improve the quality of life and standard of living of the community? Of the world? I feel like a 1950's traditional conservative when I suggest that, but most Americans are so heavily indoctrinated with corporate greed and sociopathy they'd consider that sentiment radical leftist extremism. I'm an athiest, but ya'll need jesus (the real brown socialist one). Many would argue Americas current institutional collapse is the natural result of this systemic corruption.
I wouldn't argue that America's moral standards haven't declined (significantly) but I also think it's a romanization to suggest that 1950s America was the pinnacle of morality.
Lying, misleading, and exploiting people for financial gain has been a part of the fabric of American society since the country was founded.
If we're being honest, humans everywhere have demonstrated a high capacity for this behavior since the dawn of civilization.
Can we just trash this as a marketing term? If/when AGI arrives there will be no point quibbling over competency. What we are looking at is just bad search results
Oh no! Anyway ...
Next do Amazon that is selling AI generated foraging books: - https://www.theguardian.com/technology/2023/sep/01/mushroom-...
When I was a kid it was possible to buy any foraging books from a store and they had a minimum quality. Is that so difficult to achieve? Is profiteering not punished anymore?
Well, they disclaimed and the user acknowledged
Source: https://www.youtube.com/watch?v=P-QaEB5eXSU Minute 3:00
In fact, in most EU countries "the user acknowledged" works only for a very small subset of stuff, precisely because our lawmakers know that the strong party in a contract would use that to get away from every legal obligation.
And what institution gives out the licenses for journalists and scientists? Is it revokable?
https://medium.com/@klaudibregu/google-greedy-gets-karma-8ae...
It’s super easy to catch on dates and numbers, but it gets other details wrong all the time too. But so many people won’t be double checking the results.
"rewire brains for AI dependency" (for money and power reasons obvious to everyone).
In contrast to "secretly implant an agenda about non-AI subject N", which is complicated enough that AI companies are still too out of control to be attempting yet.
Guess that's the end of their AI overviews in the EU!
Overregulation, at best, is a good way to guarantee that your country won't have access to interesting and useful features and technologies. At worst, it's a good way to guarantee that the twenty-first century will belong to the US, if not to China.
I can make mistakes. It's on you to fact-check my claims.
Do you think these are harmless statements? Does the disclaimer suffice? If I was Google's AI Overview, do you think 100% of people will check those sources?
There is nuance here, and it's not going away because AI and innovation.
Yes, I do. An assertion made without evidence can be dismissed without evidence, whether it comes from Google or from you.
If you demand perfection, you will receive nothing. Why is that so hard for people to understand? The world simply cannot work the way you say it should.
It can destroy you. You are analyzing this on the assumption that people are rational and have capacity to do the check. They have neither.
Anyone who does accept Google's AI output blindly will soon find that their mistaken opinion of CamperBob2 is the least of their problems. There is a reason Google goes (well) out of their way to warn people that the results may be wrong. That should be sufficient warning for reasonable people of good faith.
It is obvious that the defamed companies are the ones having a problem, not the ignorant viewers.
Why should those companies not hold Google liable for that outcome?
What profit? I don't know either but they enabled this for a reason right?
I very much hope we don't see attempts to re-write T&C to avoid this liability.
After all, if I can get ChatGPT or Claude to say something false that should count too, right?
The arguments of the ruling should generally apply even when the AI agent makes false statements it wasn't notified about. But in that case the defendant might have a stronger claim about not being able to reasonably ensure the correctness of all statements, and having taken reasonable measures to ensure correctness. Google couldn't really claim any of that after ignoring cease and desist letters about the false claims
Imagine a search for your name resulted in an AI summary saying you are involved with child-trafficking because low capability model linked your first name and perhaps a couple articles on supporting children non-profits to it — and then offering that in a convincing sounding summary right at the top!
Someone needs to hold the liability at the end of the day. People are experiencing real harm from false claims LLMs are spitting out.
A disclaimer and couched language will probably fly through. And it's going to matter what expectations an user could reasonably have, too.
If you knew this was all possible and you did it anyways for personal gain then you are additionally negligent which may add aggravation to your charges.
If you sell food, in a food stall, labeled as food and you add a disclaimer that it is toxic and will make you sick. You are still selling toxic food and you are liable for it.
Google is pretending to give answers to your questions. They offer you a service about answering questions. And then they add a disclaimer "we do not answer questions just write bullshit". That is still fraud and Google should be liable for it.
> isn't this just a soft ban on the deployment of non-deterministic software?
Tetris is non-deterministic and it is not banned like millions of other programs. I do not follow you.
They are also no longer covered by safe harbour provisions because it is them answering it, not some content they refer you to.
Honestly I can understand the ruling, but the side effects might be severe.
It's hard to see if the individual complaints really support a general problem, or if this simply the only result that talks about "scam + business-name". Probably, the latter.
The same problem happens on google search, if you look "<obscure false fact>", you'll get pages mentioning that false fact. If you fall into the trap of confirmation bias, it leads you to think the false fact is in true.
All three have the ability to perform a web search, then compose a reply based on the search results. Pretty much the exact thing that Google AI Overview does. This ruling may make them liable for false answers.
No, the article implies the court’s logic is that the AI search results are presented as search results and that’s a big part of why they are liable. It seems like the court (again, according to the article) does not find the disclaimers that Google has slapped on the AI results compelling because again, it chose to represent these as a summary of search results and it is aware of the failure rate.
> The court also found that the AI overview made claims "that are not even made in the search results." None of the linked sources drew any connection between the plaintiffs and the shady companies the AI mentioned. The court called these "the defendant's own statements."
> Google built the AI, Google offered it to users, so Google owns what it produces, "because it alone has influence over the AI's offering and the algorithms with which the AI operates."
Google does not, as a general rule, control the actual content of search results, but usually there’s a distinction between the ranking and presentation of the results vs. the actual content. In this case, the court is basically saying, “You sold this to people as a search summary, you know it might be full of crap and you chose to do it anyway. No, you don’t get to claim the equivalent of a US safe harbor defense.”
There is a subtle difference in stating it as a search summary compared to an opinionated answer. Most users are always going to treat it as a response from google instead of search results where the user is still responsible for understanding and come up with their own interpretation.
This is probably the right step in some sense to make one liable for their statements/assertions.
Is something like,
"People online say that x y and z because a b c"
a credible, correct answer, even if it isn't because of a/b/c?
The instance of this ruling people apparently did not actually say any of the offending claims. 'The court also found that the AI overview made claims "that are not even made in the search results." None of the linked sources drew any connection between the plaintiffs and the shady companies the AI mentioned. The court called these "the defendant's own statements."'
One way to formulate things that would be less would be "once support a time, in some fabulated world, it's not impossible that some imaginary character would say something following some reason." But then, of course this is not aligning the the deception scheme pushed by companies putting in their interface that the "machine is thinking hard for you".
Every AI model can make something up sometimes. Over millions of daily calls, it's essentially impossible for the technology to be guaranteed correct 100% of the time.
You're not even paying for a google service, search is free... You might be the product, and your data, but you didn't directly pay for a service and they didn't sell you a fake service.
I'm not taking Google's side, this isn't about whether it's right or wrong to rob websites of traffic, this is about AI's returning search metadata.
But I'm surprised that they lost this argument, and the line they took in the first place.
The Internet isn't made of fact checked data, it's crowd sourced. How can anyone be liable?
If instead Google gives you an answer right there on google.com, without going to another site, they ARE responsible for it.
That makes sense to me?
(What's the alternative, after all? Having no one responsible for what the AI summary says is clearly untenable.)
Probably not, because it's a similar situation where Wikipedia is accumulating user provided content. And people know Wikipedia can be freely edited.
You, however, might be liable. It's your content.
https://en.wikipedia.org/wiki/Asian_News_International_v._Wi...
Directly? Quite possibly. They'd then have to transfer that liability to you.
Yes, and it's called defamation when you don't follow it.
This is why you have to say "I think this person is a murderer" and not "This person is a murderer."
One is opinion. One is fact.
This isn't super hard.
I was prepared to say the same thing as you but after reading it seems totally fair.
The key difference is that this would be illegal if a human wrote it too.
None of "AI" companies call their apps "Entertainment fun text generator". They are call them serious names, use words like "intrllegence" and "thinking".
So yeah I'd think if any of "AIs" start to recommend to drink some bleach or take a flight from a 10th floor window these companies should be liable.
This is a gigantic own goal for Google. The average person’s impression is that Google AI is much worse than ChatGPT, even though that’s not actually the case. Google is shoving a terrible model in everyone’s faces.
Genuinely, by far the most common situation where claims of defamation arise in Germany is any less than 5 star review left to a business on google maps.
It's not like this exclusively affects restaurant reviews, every corner of German society is subject to this same evil censorship mechanism.
Probably not, for the same reason search results aren't an issue.
Even if the answers are correct, they could still be biased, incomplete, misleading, and all the other media-literacy things people should be looking out for.
This ruling seems to go the opposite direction; 'I am legally obliged to give correct answers, so I am always right, trust the AI'.
I first tried it to remind me of what happened in a previous book in a series that I was reading. When I realized it was either misstating plot points or straight up hallucinating, I tried it on a bunch more books to amuse myself.
Older classics are of course more accurate, but for newer or less popular books Gemini won't shy away from giving you a summary culled from misinterpreted Reddit threads and Goodreads reviews. It's like getting a secondhand account from someone who talked to another person who had read the book a long time ago. You get the general gist of it, but with some added flavor.
Even if you upload an entire epub of a book, the results aren't stellar. Rather than a Cliffs Note's quality summary, they're pretty sparse or leave out important bits of information. One chapter summary I got back made a point of describing what one of the characters was wearing, even though it had absolutely zero to do with anything else. Yes, that's technically a "summary", but not quite my tempo.
If Google wants to present summaries of websites in anything more than a very, very superficial description, they're going to have to improve their model's ability to understand context and importance. In theory, a novel is a self-contained bundle of text, so pulling accurate information out of it should be straight forward. A website is naturally going to be way more of a challenge.
All that said, I find the AI summaries from Google/Gemini to be quite useful and a time saver, but I know to always double check something if it's at all important.
Because of the same rules, German restaurants also get to pick and choose which reviews stay up. They can literally take down any specific reviews they like.
A restaurant that mostly gets 1 star reviews will still show up with 5 stars on Google maps, as they will simply delete the reviews with less than 5 stars as defamatory.
Here's a couple of examples:
https://www.reddit.com/r/germany/comments/1peujau/google_rev...
https://www.reddit.com/r/germany/comments/18z4shs/legal_thre...
https://www.reddit.com/r/frankfurt/comments/1lox7ha/bad_revi...
https://www.reddit.com/r/germany/comments/1iiaco8/restaurant...
https://www.reddit.com/r/AskGermany/comments/1ha7sxf/why_do_...
https://www.reddit.com/r/SipsTea/comments/1t613w7/it_was_fin...
https://www.reddit.com/r/germany/comments/1l98608/threatenin...
https://www.reddit.com/r/germany/comments/1sw34jc/can_you_ge...
https://www.reddit.com/r/germany/comments/1sw34jc/can_you_ge...
https://www.reddit.com/r/germany/comments/10kmn66/writing_an...
tl;dr Germans are particularly bad at coming up with reasonable rules to handle these situations
Broad defamation laws always have overwhelmingly negative consequences.