{"id":178,"date":"2025-12-10T04:00:20","date_gmt":"2025-12-10T04:00:20","guid":{"rendered":"https:\/\/pcbottleneckcalculator.net\/news\/?p=178"},"modified":"2025-12-10T04:00:20","modified_gmt":"2025-12-10T04:00:20","slug":"ai-as-the-new-electricity-bill-why-model-costs-will-decide-which-startups-live-or-die","status":"publish","type":"post","link":"https:\/\/pcbottleneckcalculator.net\/news\/ai-as-the-new-electricity-bill-why-model-costs-will-decide-which-startups-live-or-die\/","title":{"rendered":"AI as the New Electricity Bill: Why \u2018Model Costs\u2019 Will Decide Which Startups Live or Die"},"content":{"rendered":"<p><span style=\"font-weight: 400;\">In the 20th century, energy efficiency was one of the most important factors determining whether a factory would make a profit: a higher-than-expected electricity bill could lead to a serious financial crisis, halting all production. In the 2010s, this was replaced by \u201ccloud costs\u201d. A very high AWS bill, for example, could be the death knell for a startup, a problem that even affected sites like the <\/span><a href=\"https:\/\/fortunica.org.uk\/\" target=\"_blank\" rel=\"noopener\"><span style=\"font-weight: 400;\">Casino Fortunica<\/span><\/a><span style=\"font-weight: 400;\">.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Now, these have been replaced by \u201cAI model costs\u201d, and no, this isn&#8217;t related to the &#8220;training costs&#8221; of the model used. The problem with AI models is recurrence: many minor design issues can lead to extremely large operating expenses. Let&#8217;s take a closer look at this and see how it affects startups.<\/span><\/p>\n<h2><span style=\"font-weight: 400;\">Cloud Costs vs Model Costs\u00a0<\/span><\/h2>\n<p><span style=\"font-weight: 400;\">In the cloud model, costs were simply tied to the number of users. A startup offering services via the cloud would have to migrate to a larger infrastructure as the number of users increased, which would increase bills. An unexpected decrease in the number of users or a sudden increase in service fees could become a serious profitability problem.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">The problem with the AI \u200b\u200bmodel, however, isn&#8217;t the number of users. Even a single user can strain the infrastructure by as many as 100 users, because this model differs from others in the following ways:<\/span><\/p>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">The load on the infrastructure is determined by user actions, not the number of users.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Even a single user can make hundreds of calls in a very short time, especially if they&#8217;re chatting with the AI.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Unfortunately, this is unpredictable, as each user&#8217;s interaction is different from the next.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">For the same reason, the actual cost is determined by the content, UX design, and token <\/span><span style=\"font-weight: 400;\">length.<\/span><\/li>\n<\/ul>\n<table>\n<tbody>\n<tr>\n<td><b>WHAT IS TOKEN LENGTH IN AI?<\/b><\/td>\n<td><span style=\"font-weight: 400;\">Every question you ask an AI or every text you write consists of a certain number of characters. Roughly, every 4 characters correspond to 1 token. A paragraph of 75-80 words is approximately 100 tokens. The more tokens, the more processing power the AI \u200b\u200buses to understand, process, and answer that query.<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>&nbsp;<\/p>\n<p><span style=\"font-weight: 400;\">Even a single sloppy prompt can inflate the bill unexpectedly. In the traditional cloud model, startups don&#8217;t need to worry about service costs until the number of users reaches a certain threshold, and by then, they&#8217;ve already reached a certain level of profitability. In the AI \u200b\u200bmodel, the situation is the opposite: from day one, your infrastructure can be strained beyond expectations. Even a startup with just 200 users may have to burn through thousands of dollars every month before it can even become profitable.<\/span><\/p>\n<h2><span style=\"font-weight: 400;\">The Problem Is Not the Training Costs<\/span><\/h2>\n<p><span style=\"font-weight: 400;\">Training an AI model can be expensive, but that&#8217;s not the problem. An AI startup will use an already trained model. Even if it wants to train its own model, it does so before launching its services, so these costs are already part of the planned budget\u2014they&#8217;re not unexpected.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">The problem begins when users start interacting with AI. Even every click a user makes carries a cost. If it&#8217;s not properly set up or configured, these costs will increase even further. For example, if a &#8220;premium&#8221; AI model is used even for trivial tasks, the infrastructure load increases further.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Unfortunately, this isn&#8217;t a problem with a simple solution. The classic engineering dilemma is simple: fast, cheap, and good\u2014you can only choose two; you can&#8217;t have them all. In AI, this dilemma becomes the following: model quality, low latency, low cost\u2014you can only choose two.<\/span><\/p>\n<table>\n<tbody>\n<tr>\n<td><b>WHAT IS LATENCY IN AI?<\/b><\/td>\n<td><span style=\"font-weight: 400;\">This is simply the response time to user queries. It determines how quickly you&#8217;ll get a response when you ask AI something.<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p><span style=\"font-weight: 400;\">So, if you want a model as high quality as ChatGPT and a response time of less than one second, you&#8217;ll have to bear the cost. You could also opt for low costs and low latency, but then the model quality would be subpar and wouldn&#8217;t deliver high-quality results.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">There may be no simple solution to this problem, but there is perhaps a way around it: innovative and creative startups can do this with &#8220;model routing.&#8221;<\/span><\/p>\n<h2><span style=\"font-weight: 400;\">Load Balancing via Model Routing<\/span><\/h2>\n<p><span style=\"font-weight: 400;\">In the early 2000s, web developers invented many tricks to ensure servers could handle incoming requests. These were used for &#8220;load balancing,&#8221; meaning they used clever tactics to reduce user workload and keep the server running. A similar balancing technique can be applied to AI: this is called &#8220;model routing.&#8221; We can explain what model routing is with examples:<\/span><\/p>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Using a small and inexpensive model to identify keywords.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Switching to a mid-tier model for classification.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Using a premium model for complex reasoning.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Leveraging cached results for repetitive tasks.<\/span><\/li>\n<\/ul>\n<p><span style=\"font-weight: 400;\">The biggest problem AI startups face is using the same model for every type of query. This means both very simple and very complex tasks are often solved using the same model, creating a never-ending workload and increasing costs.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">In a building with 100 employees, you don&#8217;t need to heat the entire building just because one person is cold\u2014this would be prohibitively expensive. Simply placing a heater next to that employee would solve the problem cost-effectively. AI model routing is based on this very principle.\u00a0<\/span><\/p>\n<h2><span style=\"font-weight: 400;\">Tokens Are the New Kilowatts<\/span><\/h2>\n<p><span style=\"font-weight: 400;\">We mentioned above that one of the biggest problems for businesses in the 20th century was electricity bills. Tokens, in this sense, constitute the &#8220;kilowatt-hours&#8221; of the AI \u200b\u200beconomy. Each prompt creates a varying number of tokens, and electricity is used to process them. This means that the fewer tokens, the lower the electricity costs.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Unmanaged, token costs can reach brutal levels. Therefore, token optimization could be another solution to the problems faced by the AI \u200b\u200bmodel. This solution could even be so effective that it could create a new field of work in its own right: in the near future, a profession called \u201ccost engineering\u201d could emerge, focusing solely on token optimization.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">This profession is different from \u201cprompt engineering\u201d. For the past few years, almost every AI startup has been hiring prompt engineers, and their primary goal is to make AI &#8220;smarter.&#8221; However, as mentioned above, smarter means higher costs. A cost engineer, on the other hand, would focus on keeping the system smart without additional costs: this could become one of the most in-demand professions in the near future.<\/span><\/p>\n<h2><span style=\"font-weight: 400;\">Simple UX, Less Load<\/span><\/h2>\n<p><span style=\"font-weight: 400;\">Another solution is to make better UX choices. UX design can be so complex that it drives up utility bills. This is because every user interaction has a cost within the AI \u200b\u200bsystem. So, if a user can perform a certain action in 5 clicks instead of 1, the cost to the system is 5x higher. Most AI startups don&#8217;t pay much attention to UX design and simply want it to look &#8220;as impressive as possible.&#8221; However, a UX that is &#8220;as simple as possible&#8221; can significantly reduce costs.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">In a market where model costs are paramount to success, the winners won&#8217;t be the startups with the flashiest interfaces or the highest-quality AI models. On the contrary, startups that think about model routing, use token optimization techniques, and design their UX designs to create as little load on the system as possible will succeed in surviving.<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>In the 20th century, energy efficiency was one of the most important factors determining whether a factory would make a profit: a higher-than-expected electricity bill could lead to a serious financial crisis, halting all production. In the 2010s, this was replaced by \u201ccloud costs\u201d. A very high AWS bill, for example, could be the death &#8230; <a title=\"AI as the New Electricity Bill: Why \u2018Model Costs\u2019 Will Decide Which Startups Live or Die\" class=\"read-more\" href=\"https:\/\/pcbottleneckcalculator.net\/news\/ai-as-the-new-electricity-bill-why-model-costs-will-decide-which-startups-live-or-die\/\" aria-label=\"Read more about AI as the New Electricity Bill: Why \u2018Model Costs\u2019 Will Decide Which Startups Live or Die\">Read more<\/a><\/p>\n","protected":false},"author":14,"featured_media":179,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[7],"tags":[],"class_list":["post-178","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-sports"],"_links":{"self":[{"href":"https:\/\/pcbottleneckcalculator.net\/news\/wp-json\/wp\/v2\/posts\/178","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/pcbottleneckcalculator.net\/news\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/pcbottleneckcalculator.net\/news\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/pcbottleneckcalculator.net\/news\/wp-json\/wp\/v2\/users\/14"}],"replies":[{"embeddable":true,"href":"https:\/\/pcbottleneckcalculator.net\/news\/wp-json\/wp\/v2\/comments?post=178"}],"version-history":[{"count":1,"href":"https:\/\/pcbottleneckcalculator.net\/news\/wp-json\/wp\/v2\/posts\/178\/revisions"}],"predecessor-version":[{"id":180,"href":"https:\/\/pcbottleneckcalculator.net\/news\/wp-json\/wp\/v2\/posts\/178\/revisions\/180"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/pcbottleneckcalculator.net\/news\/wp-json\/wp\/v2\/media\/179"}],"wp:attachment":[{"href":"https:\/\/pcbottleneckcalculator.net\/news\/wp-json\/wp\/v2\/media?parent=178"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/pcbottleneckcalculator.net\/news\/wp-json\/wp\/v2\/categories?post=178"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/pcbottleneckcalculator.net\/news\/wp-json\/wp\/v2\/tags?post=178"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}