“Look, Python is way easier to use than other languages! Look how complex this easy task is in Python versus other languages like assembly and brainfuck!”
I’m not saying “do stuff in C it’s easier than Python”, but if I took e.g. C# then it’s also just two lines. That supports everything and is also faster than the Python implementation.
I mean, is it? I personally haven’t found Python using much less boilerplate. It’s possible, but you end up with something inflexible that’s hard to maintain.
I was friends with her and I trusted that she would handle it well if she wasn’t interested. And turns out she indeed wasn’t interested, but we did talk about it and decided to just stay friends. It was a little awkward as my feelings for her still lingered a bit, but eventually that passed and I’m now with a wonderful girl who I think is a much better match for me.
We’re still friends to this day.
I meant a library unknown to me specifically. I do encounter hallucinations every now and then but usually they’re quickly fixable.
It’s made me a little bit faster, sometimes. It’s certainly not like a 50-100% increase or anything, maybe like a 5-10% at best?
I tend to write a comment of what I want to do, and have Copilot suggest the next 1-8 lines for me. I then check the code if it’s correct and fix it if necessary.
For small tasks it’s usually good enough, and I’ve already written a comment explaining what the code does. It can also be convenient to use it to explore an unknown library or functionality quickly.
Yes, but at least there they still use “Earth time”, just slowed down. For the moon it gets a little bit more complicated I guess.
Time moves at a different speed due to the moon’s reduced gravity. It’s not just the length of a day.
RFCs aren’t really law you know. They can deviate, it just means less compatibility.
What they didn’t prove, at least by my reading of this paper, is that achieving general intelligence itself is an NP-hard problem. It’s just that this particular method of inferential training, what they call “AI-by-Learning,” is an NP-hard computational problem.
This is exactly what they’ve proven. They found that if you can solve AI-by-Learning in polynomial time, you can also solve random-vs-chance (or whatever it was called) in a tractable time, which is a known NP-Hard problem. Ergo, the current learning techniques which are tractable will never result in AGI, and any technique that could must necessarily be considerably slower (otherwise you can use the exact same proof presented in the paper again).
They merely mentioned these methods to show that it doesn’t matter which method you pick. The explicit point is to show that it doesn’t matter if you use LLMs or RNNs or whatever; it will never be able to turn into a true AGI. It could be a good AI of course, but that G is pretty important here.
But it’s easy to just define general intelligence as something approximating what humans already do.
No, General Intelligence has a set definition that the paper’s authors stick with. It’s not as simple as “it’s a human-like intelligence” or something that merely approximates it.
Yes, hence we’re not “right around the corner”, it’s a figure of speech that uses spatial distance to metaphorically show we’re very far away from something.
Not just that, they’ve proven it’s not possible using any tractable algorithm. If it were you’d run into a contradiction. Their example uses basically any machine learning algorithm we know, but the proof generalizes.
Our squishy brains (or perhaps more accurately, our nervous systems contained within a biochemical organism influenced by a microbiome) arose out of evolutionary selection algorithms, so general intelligence is clearly possible.
That’s assuming that we are a general intelligence. I’m actually unsure if that’s even true.
That doesn’t mean they’ve proven there’s no pathway at all.
True, they’ve only calculated it’d take perhaps millions of years. Which might be accurate, I’m not sure to what kind of computer global evolution over trillions of organisms over millions of years adds up to. And yes, perhaps some breakthrough happens, but it’s still very unlikely and definitely not “right around the corner” as the AI-bros claim (and that near-future thing is what the paper set out to disprove).
Haha it’s good that you do though, because now there’s a helpful comment providing more context :)
I was more hinting at that through conventional computational means we’re just not getting there, and that some completely hypothetical breakthrough somewhere is required. QC is the best guess I have for where it might be but it’s still far-fetched.
But yes, you’re absolutely right that QC in general isn’t a magic bullet here.
The actual paper is an interesting read. They present an actual computational proof, stating that even if you have essentially infinite memory, a computer that’s a billion times faster than what we have now, perfect training data that you can sample without bias and you’re only aiming for an AGI that performs slightly better than chance, it’s still completely infeasible to do within the next few millenia. Ergo, it’s definitely not “right around the corner”. We’re lightyears off still.
They prove this by proving that if you could train an AI in a tractable amount of time, you would have proven P=NP. And thus, training an AI is NP-hard. Given the minimum data that needs to be learned to be better than chance, this results in a ridiculously long training time well beyond the realm of what’s even remotely feasible. And that’s provided you don’t even have to deal with all the constraints that exist in the real world.
We perhaps need some breakthrough in quantum computing in order to get closer. That is not to say that AI won’t improve or anything, it’ll get a bit better. But there is a computationally proven ceiling here, and breaking through that is exceptionally hard.
It also raises (imo) the question of whether or not we can truly consider humans to have general intelligence or not. Perhaps we’re not as smart as we think we are either.
I won’t pretend I understand all the math and the notation they use, but the abstract/conclusions seem clear enough.
I’d argue what they’re presenting here isn’t the LLM actually “reasoning”. I don’t think the paper really claims that the AI does either.
The CoT process they describe here I think is somewhat analogous to a very advanced version of prompting an LLM something like “Answer like a subject matter expert” and finding it improves the quality of the answer.
They basically help break the problem into smaller steps and get the LLM to answer smaller questions based on those smaller steps. This likely also helps the AI because it was trained on these explained steps, or on smaller problems that it might string together.
I think it mostly helps to transform the prompt into something that is easier for an LLM to respond accurately to. And because each substep is less complex, the LLM has an easier time as well. But the mechanism to break down a problem is quite rigid and not something trainable.
It’s super cool tech, don’t get me wrong. But I wouldn’t say the AI is really “reasoning” here. It’s being prompted in a really clever way to increase the answer quality.
It’s not a direct response.
First off, the video is pure speculation, the author doesn’t really know how it works either (or at least doesn’t seem to claim to know). They have a reasonable grasp of how it works, but what they believe it implies may not be correct.
Second, the way O1 seems to work is that it generates a ton of less-than-ideal answers and picks the best one. It might then rerun that step until it reaches a sufficient answer (as the video says).
The problem with this is that you still have an LLM evaluating each answer based on essentially word prediction, and the entire “reasoning” process is happening outside any LLM; it’s thinking process is not learned, but “hardcoded”.
We know that chaining LLMs like this can give better answers. But I’d argue this isn’t reasoning. Reasoning requires a direct understanding of the domain, which ChatGPT simply doesn’t have. This is explicitly evident by asking it questions using terminology that may appear in multiple domains; it has a tendency of mixing them up, which you wouldn’t do if you truly understood what the words mean. It is possible to get a semblance of understanding of a domain in an LLM, but not in a generalised way.
It’s also evident from the fact that these AIs are apparently unable to come up with “new knowledge”. It’s not able to infer new patterns or theories, it can only “use” what is already given to it. An AI like this would never be able to come up with E=mc2 if it hasn’t been fed information about that formula before. It’s LLM evaluator would dismiss any of the “ideas” that might come close to it because it’s never seen this before; ergo it is unlikely to be true/correct.
Don’t get me wrong, an AI like this may still be quite useful w.r.t. information it has been fed. I see the utility in this, and the tech is cool. But it’s still a very, very far cry from AGI.
This is true, but it’s specifically not what LLMs are doing here. It may come to some very limited, very specific reasoning about some words, but there’s no “general reasoning” going on.
The difference between ban and suspend isn’t a temporal difference. Here’s the Cambridge dictionary definition of “suspend”:
Here’s the definition for “ban”:
The difference between the two is the subject: an active process or service can be suspended, but something specific (e.g. an action, object or person) can be banned. Ban also implies a more official act in order to punish someone or prevent something (Johnny was banned from entering the bus), whereas a suspension doesn’t necessarily have that ‘negative’ context (e.g. the bus service was suspended, which doesn’t imply this happened because the bus driver was drunk or something).
In a more Lemmy-specific context, you could say you suspended someone’s access to the platform, or that you banned them from the platform. Neither way of saying it implies anything about the duration. You can’t however really say you suspended someone from the platform, that doesn’t really work.
In this context, I think the direct implication that a ban is handed out because someone did something bad is a lot clearer than when you use the word suspension. Because of that I believe ban to be the more context-appropriate word here. Suspend does not carry that connotation as something can be suspended for a whole host of reasons, none of which have to be related to rule-breaking. For example, federation with another instance could be suspended temporarily until the other instance does (or doesn’t do) something that is required for technical reasons.