FaceDeer

It’s actually cromulent technical terminology to call those extra degrees of freedom “dimensions”, it’s only in common parlance that “dimension” is restricted specifically to spatial dimension. Having hundreds or even thousands of dimensions is not unknown in data science.

Also, what do you mean by synthetic data? If it’s made by AI, that’s how collapse happens.

But that’s exactly my point. Synthetic data is made by AI, but it doesn’t cause collapse. The people who keep repeating this “AI fed on AI inevitably dies!” Headline are ignorant of the way this is actually working, of the details that actually matter when it comes to what causes model collapse.

If people want to oppose AI and wish for its downfall, fine, that’s their opinion. But they should do so based on actual real data, not an imaginary story they pass around among themselves. Model collapse isn’t a real threat to the continuing development of AI. At worst, it’s just another checkbox that AI trainers need to check off on their “am I ready to start this training run?” Checklist, alongside “have I paid my electricity bill?”

The problem with curated data is that you have to, well, curate it, and that’s hard to do at scale.

It was, before we had AI. Turns out that that’s another aspect of synthetic data creation that can be greatly assisted by automation.

For example, the Nemotron-4 AI family that NVIDIA released a few months back is specifically intended for creating synthetic data for LLM training. It consists of two LLMs, Nemotron-4 Instruct (which generates the training data) and Nemotron-4 Reward (which curates it). It’s not a fully automated process yet but the requirement for human labor is drastically reduced.

the only way to guarantee training data isn’t from its own model is to make it yourself

But that guarantee isn’t needed. AI-generated data isn’t a magical poison pill that kills anything that tries to train on it. Bad data is bad, of course, but that’s true whether it’s AI-generated or not. The same process of filtering good training data from bad training data can work on either.

It’s not wrong for either to draw inspiration from the other. It’s the hypocrisy that’s wrong.

I’ve made similar points in the past in discussions about robot soldiers going to war. There’s an upside to these things that people insist on overlooking; they follow their programming. If you program a robot soldier to never shoot at an ambulance, then it will never shoot at an ambulance even if it’s having a really bad day. Same here, if the security robot has been programmed never to leave the public sidewalk then it’ll never leave the public sidewalk.

It’s always possible for these sorts of things to be programed to do the wrong things, of course. But at least now we have the ability to audit that sort of thing.

Are you suggesting that the same amount of crime is happening but they’re deciding not to report it because there’s a robot there? That’s the measure they’re touting, the reduction in crime reports.

You joke, but presumably that’s when it recharges.

They’re not both true, though. It’s actually perfectly fine for a new dataset to contain AI generated content. Especially when it’s mixed in with non-AI-generated content. It can even be better in some circumstances, that’s what “synthetic data” is all about.

The various experiments demonstrating model collapse have to go out of their way to make it happen, by deliberately recycling model outputs over and over without using any of the methods that real-world AI trainers use to ensure that it doesn’t happen. As I said, real-world AI trainers are actually quite knowledgeable about this stuff, model collapse isn’t some surprising new development that they’re helpless in the face of. It’s just another factor to include in the criteria for curating training data sets. It’s already a “solved” problem.

The reason these articles keep coming around is that there are a lot of people that don’t want it to be a solved problem, and love clicking on headlines that say it isn’t. I guess if it makes them feel better they can go ahead and keep doing that, but supposedly this is a technology community and I would expect there to be some interest in the underlying truth of the matter.

No, researchers in the field knew about this potential problem ages ago. It’s easy enough to work around and prevent.

People who are just on the lookout for the latest “aha, AI bad!” Headline, on the other hand, discover this every couple of months.

AI already long ago stopped being trained on any old random stuff that came along off the web. Training data is carefully curated and processed these days. Much of it is synthetic, in fact.

These breathless articles about model collapse dooming AI are like discovering that the sun sets at night and declaring solar power to be doomed. The people working on this stuff know about it already and long ago worked around it.

This is “technology news and articles?”

Seems like this place is increasingly just people yelling at AI-generated clouds.

But at least that crappy bug-riddled code has soul!

It’s almost doublethink, people celebrating how the Fediverse is an open protocol for sharing public discussion and then going surprised-Pikachu at the notion that public discussion might be viewed by someone the don’t want to view it.

If you don’t mean for something to be public, don’t post it on a public forum.

I’m Canadian so I’m not a voter in the contest you’re presenting, but if I were I would vote Democrat. And of the trio you present for the Democrats, I would say that the position I’d compromise on would be gun control. Not because American gun culture isn’t bananas and it’s not a serious problem, but because I can’t see any plausible way to fix it in the short term. So might as well let it go for now and deal with the more important stuff that affects more people.

I think a more reasonable compromise would be to give Republicans most of what they want on immigration reform. That seems to be something they consider to be of critical importance, but that I think can be allowed without it causing significant harm. If the American economy starts to suffer as a result of not having illegal immigrant workers then that will be motivation for further reforms. I think it’s important to have the laws try to reflect the realities, though, and having the economy literally depend on large-scale lack of adherence to the law of the land is a bad place to be. Just make sure not to be monstrous about it - don’t do the concentration-camps-for-children thing, try to maintain basic asylum access for those who truly need it, and so forth.

Not in every way. They’re cheaper and faster.

The suits aren’t technically needed for reentry, since the capsule isn’t supposed to be depressurized at any point during the trip. It’s just another layer of “if something goes wrong.” So if it’s a choice of taking that risk or staying on an exploding ISS you go with the risk. I expect that even if the suit can’t be connected to Dragon’s umbilicals it could still be sealed for at least a few minutes of air during the riskiest bits of the trip.

As with most safety procedures, it’s written in blood.

There is an alternative, in the event of disaster there’s room on board the Dragon capsule currently docked at the station for them to come back down. They’d be strapped into the cargo hold rather than a seat, but that’s acceptable in a disaster situation.

They’re professional astronauts who have worked their whole lives for the opportunity to get into space. Both Butch and Sunny were probably doing the last mission of their career with this trip, so having it extended from 8 days to 8 months could well be a dream come true for them.

And yet this community seems more techno-pessimistic than even /r/technology, which is a challenge.

Indeed. And Boeing is the main contractor for it so you can be sure it won’t suffer any mishaps.

There’s others that are trying, Blue Origin has their New Shepherd rocket that is able to land, but it’s a suborbital tourism vehicle that’s basically just a toy. They’re working on a partly-reusable orbital launcher that’s like a souped up Falcon 9 but it’s still in development. Several other smaller startups are working on smaller Falcon-9-like launchers with expendable second stages, and China is building a straight up carbon-copy of the Falcon 9 and Starship. But SpaceX is the leader in this field and currently the only one who’s actually successful. Everyone is following in their wake at the moment.