I found the book opening with "A Message from the Emperor" very befitting, as it seemed to me that this story, in fact just a single page, nicely condensed the tone of the entire collection. In my opinion the stories explore the common theme of irreconcilable discrepancies (among human beings). "A Message from the Emperor" in particular depicts a person, a "contemptible subject" of the Emperor, waiting for a message from the Emperor that will never arrive. He knows for certain that the message won't arrive; yet he still waits.
In the well-known Metamorphosis, the discrepancies take on a physical form -- the protagonist, Gregor Samsa, finds himself turned into a bug one morning. While Samsa attempts his best to convey what he thinks to his family, the members of family, understandably, are incapable of even conceiving that this bug, Samsa, may have any intellect. Communication between these two parties is broken beyond repair; the present discrepancies are irreconcilable.
Likewise in "In the Penal Colony," and in "A Hunger Artist." In the former a foreigner is made to judge whether it is right to ban a particular execution machine of the past. The last remaining advocate of the machine, an army officer, tries his best to highlight the merits of it. He goes through great pain explaining how each and every piece of the machine works with great affection. No, the foreigner wont be deterred. The foreigner is as foreign as one could be from the idea of cruel execution. The hunger artist's vocation is to fast. He fasts in public and receives compensation from the spectators. He takes great pride in what he does; he only stops fasting because the convention prohibits him from going on. But his is a dying occupation. People gradually lose interest. How is it possible to convey to those ignorant people what noble a deed it is to fast?
Remember the time you felt deep despair for not being able to get through to someone you care for (when no matter what you say just won't mean the same thing to you as to the other person)? Albeit in varying contexts, it is this devastation that Kafka so masterfully depicts in these stories.