The AI ​​had a problem with its hands. Now that he’s fixed it, we’re in trouble

Donald Trump is on the ground, on his knees, tackled by cops. The picture is fake – as has been widely documented – yet the photorealism of this shot, generated byartificial intelligencehas misled many.

Trump’s fake snap under arrest won’t be the first – or even the last – to throw us into crisis. But so far, fortunately, the AI ​​that produces these kinds of images – it can be Midjourney but also Stable Diffusion or Dall-E 2 – I had a very specific weak point: the hands.

A series of images, which went viral last January, highlighted the problem well. Miles Zimmerman, a programmer from San Francisco, used Midjourney to create shots of a party that never took place, with totally made-up people. Yet no one (or almost) would have doubted the veracity of the photos at first sight, if it wasn’t for the hands of the girls portrayed. One of them had at least ten fingers. Another, on the other hand, had five but of an unnatural length.

These shots have triggered ironic comments among Twitter users. And they have reassured, in a certain sense, those who were already predicting a world overwhelmed by deepfakesAI-created images in which – for example – a person’s face is seamlessly replaced by that of another individual.

By and large, AI-generated images look consistent and plausible. But often, as one goes deeper with the gaze, and shifts the attention to elements in the background, or around the main subject, one realizes that these appear distorted and deformed.

It is evident in the fake photo where Trump “is arrested”. The author of the images, the journalist – and founder of Bellingcat – Eliot Higginshe underlined it with a very crude tweet, reproaching a certain ‘stupidity’ to those who interpreted the shot as true: just look at the chaotic symbols on the policemen’s uniforms, but also (and above all) two hands that appear between the agents, that they have very little human.

Midjourney (and similar) AI has been trained on billions of images and their text description. In this way the algorithm has learned that the word “astronaut” corresponds to the image of a man or a woman wearing a suit and a space helmet. So when a user, using an interface similar to a chat, asks the AI ​​to generate a new image that predicts the presence of an astronaut, the software goes to fish the images related to that description. AND starting from a random shapecomes as close as possible to the user’s request through a refinement that involves many iterations, ie finishing passes that increase the accuracy and quality of the final image.

We have seen that this process is miraculousto the point of winning art competitions and shaking photographers, designers and illustrators, whose work is suddenly at risk.

Yet the AI, so good at generating the faces of human beings, so unique and full of expressions and particularities, he has serious problems with his hands. Why?

A BuzzFeed reporter asked Stability AI, the company it launched in August 2022 Stable Diffusionone of the AIs text-to-image strongest out there. A spokesperson for the company replied that “in the volume of data available to AI it is universally known that hands are more visible than faces. And which, moreover, are much smaller in the original image”. The algorithm, therefore, has more difficulty studying and replicating them.

Italian Tech addressed the same question to ChatGptthe popular artificial intelligence developed by Open AI, and the answer is strikingly similar: “True, human faces also have numerous unique features, such as facial proportions, facial expressions, wrinkles, hair, and so on. However, AI seems to have less trouble generating human faces than hands because human faces have some common features that make them easier to generate. For example, the symmetry of the human face makes it easier for the image generation algorithm to create a realistic image of the face, since the two sides of the face are usually very similar. Furthermore, there are also many more images of human faces available than hands, which means that the image generation algorithm has more examples to learn from.

What ChatGpt “affirms” has its own logic and above all has a foundation of truth. The more data an algorithm has at its disposal, the smarter it is. And with regard to the faces, it is true that these are more easily attributable to certain ‘patterns’, i.e. a relevant regularities that constitute a model: the arrangement of the eyes, for example, or that of the nose or mouth.

After all, one of the first artificial intelligence-based facial recognition systems was based on these patterns, developed in the mid-1990s by the US Department of Defense with the help of thousands of photos of its employees.

In short, so far it has been relatively easy to unmask an AI-created image, although it was credible: it was enough to observe the hands, count the fingers and check their shape. Another fake photo of Trump is circulating, blocked by two agents, in which a third policeman rests his hand on the left fist of the former US president: well, the policeman’s hand has something strange, it seems to have no index.

However even these defects are destined to disappear. Indeed, in many cases they seem to have already disappeared.

The latest Midjourney update, occurred last March 16, seems to have solved the problem of the hands and fingers. A Twitter user, Zein Khan, underlined with two images how the new version of AI is capable of generating believable hands: The girls portrayed were invented by AI and have exactly five fingers. Five, the same number that identifies the Midjourney update.

The progress of the AI ​​was also highlighted by a young Midjourney user interviewed by the Washington Post, the nineteen year old Aidan Ragan, who compared the hands generated by Midjourney v4 (on the left, in the tweet below) with those created following the latest update (right, in the same tweet).

The difference is striking. Ragan said he already misses the old ‘imperfections’, which he believes are the result of a fascinating artistic interpretation of the AI. “Now, however, the model seems very rigid – said the programmer – everything is more robotic, it is reduced to a tool”.

“Before sorting out these details – the computer forensics professor Hany Farid, from the University of California – average users counted their fingers and came to the conclusion that, perhaps, a certain image was fake. But now that the details of the hands are also corrected, these visual evidences will start to be rarer”.

The same concern stirs Del Walker, specialist in creating 3D characters, which just last March 16 – the day of the Midjourney update – published images in a viral tweet, seen by more than 1 million users, in which it stated: “Midjourney can now create hands in the right way. Be critical of any politically motivated images you see online.”

“The mistake you shouldn’t make is to say ‘look at your hands’ or ‘look at your teeth’ [che nelle foto generate dall’IA sono solitamente più numerosi rispetto alla realtà, ndr] – wrote Walker – This artificial intelligence has only been producing photos for four months. In two years it will be indistinguishable. The correct thing to ask is: ‘Does this photo cite real sources?’ or ‘Can we prove the origin of this image?’”.

Any artist or illustrator knows that the hand is perhaps the most complex body part to draw: for the many moving parts with respect to the rest of the body, for example, and for the overlapping that occurs when the fingers assume different positions.

Not surprisingly until the Middle Ages the representation of the hands did not shine for its realism. Then came the Renaissance. And Leonardo Da Vinci, with his drawings that highlight the bones and ligaments of the hand. And Michelangelo, Botticelli and Dürer. Even the hand has become art.

In just four months, the machines have assimilated 623 years of history. And they are preparing to leave their mark on the art that man will produce in the future.