Economic Observer reporter Liu Sixuan

The integral consumption is becoming the customer complaint focus of the button (Coze) of the AI Agent product under ByteDance.

On the consumer service platform "Black Cat Complaints", using the keyword "Button Points" for search, relevant complaints have appeared successively since April and continued to be added until September, with a total of over 220, involving abnormal consumption of points, deduction of points after task suspension, and inability to contact customer service.

Packaging Token consumption into points is a common billing method used by most AI Agent products currently. Under such billing logic, users neither know how many steps a task will be split into by the agent for execution, nor can they calculate how many tokens are burned for each step.

When the agent breaks down calling the model, executing tasks, and repeatedly trying again into a series of invisible actions, the points purchased by the user with money fall into a black box.

On October 10th, in response to issues such as point consumption, Button Customer Service reported that if abnormal point consumption occurs due to product problems, users can submit a work order through the corresponding portal on the official website. The staff will handle it gradually within 5 to 10 working days and issue point compensation to affected users. In addition, Button plans to launch a "self-service refund" function in the near future. Member orders purchased by users on Button's official website and Android App channels will soon support self-service refund of unused balances.

Running tasks is like opening a blind box

A button user told Economic Observer that since June, he has been using buttons almost every day to perform timed tasks. Without any adjustments to the task description, the difference in daily consumption of points is huge, with a minimum consumption of less than 100 points and a maximum consumption of over 20000 points.

The above experience is not an isolated case. On the Black Cat Complaints Platform, a paying account claimed that a single task consumed nearly 50000 points, directly burning out the entire month's points quota; Another user subscribed to a personal premium package, which is officially priced at 99 yuan per month and includes 99000 points. Due to the Agent executing tasks without user confirmation, all points were exhausted within two days; Another user encountered structural damage to the platform's material library during the production of an AI short video project, causing the backend to continue running idle and consuming over 170000 points.

In the above complaints, users also mentioned another issue - it is difficult to find real customer service. There is no obvious customer service entrance on the button page. After the user clicks on the help center, the page will pop up an AI customer service ("Button Assistant") window, informing the user that "the free version does not currently support submitting work order feedback". Personal paying users can submit work orders and wait for customer service response; Among individual users, only those who subscribe to the Personal Premium Edition for 999 yuan per month can obtain real-time manual support during working hours through online customer service.

Some users reported that they only received a response from customer service after submitting their work order, which was similar to the content generated by the button assistant. In addition to work orders, users can also join communication groups if they want to contact the official website. However, the instructions for communication groups clearly state that there is no on duty customer service within the community.

Why does token consumption fluctuate

In natural language processing, Token is the smallest unit of text processed by a model, typically equivalent to 0.75 English words or half a Chinese character per Token. In pure API call mode, the token should be labeled with clear input and output words.

But AI agents are not simply chatbots. Agents have the ability to autonomously plan, call tools, and perform logical reasoning.

It is difficult for ordinary users to estimate the cost of a complex task in advance: why do token consumption differ by a hundred times for the same scheduled task?

Firstly, information retrieval is uncontrollable. Dealing with relatively vague timed tasks, such as information retrieval, the amount of information crawled by the agent each time is not fixed and the length of the information is inconsistent, which has a significant impact on token consumption. ”A developer explained to the Economic Observer the possible reasons for the fluctuation in consumption of the same task.

Taking the "Retrieve Today's News" command received by the Agent as an example, the amount of information captured by the search plugin it calls determines the amount of tokens consumed. These pieces of information will be stuffed into the context window of the large model for reading and summarizing. The user sees the same output sentence, but below the water surface, the reading volume of the model varies greatly.

Secondly, multiple rounds of loop inference and backend invalid retries will significantly increase token consumption. A research and development personnel from a leading Internet company explained that the greatest charm of Agent lies in its "autonomy". When executing complex tasks, if an API interface returns an incorrect data format or the crawler encounters anti crawling mechanisms, the agent will "think": "I have encountered an error, I need to try again in a different way. If a strict maximum retry count hard circuit breaker mechanism is not set in the underlying code, the agent will fall into a dead loop in the sandbox. Every retry, it will bring all previous failed conversation history to request the model, and the consumption of tokens will skyrocket exponentially.

The third is the infinite stacking of ultra long contexts. The R&D personnel of the head Internet company added that many ordinary users are used to chatting in the same dialog box all the time. But the memory mechanism of the large model determines that for every sentence currently sent by the user, the system will package and resend the entire conversation history of dozens of previous conversations. The longer the conversation, the higher the implicit cost of each interaction.

The above-mentioned developers also mentioned that in AI Agent products, if users do not explicitly specify a model, or the product does not support selecting specific models, the platform may route different models based on task difficulty, which can also result in different token consumption for the same task, especially for models recently released by various vendors, which consume significantly more tokens when processing the same task compared to older models.

Beyond the technical level, the platform operation mechanism exacerbates the opacity of point consumption: the official document of Button clearly states that for tasks that fail due to button platform technical issues, the platform will provide periodic point compensation, "automatically recognized and processed by the system without manual application", but does not introduce how to handle failed tasks that the system did not recognize or whether there is a manual review mechanism. Multiple users reported on Black Cat's complaint that users who consumed idle due to damage to the material warehouse or sand box collapse have not received any compensation, and their email communication still received template style responses.

The user issue has been unresolved for a long time or is related to previous organizational adjustments. A person close to the button team told the Economic Observer that the "Doubao Work" was released at the end of August, and the button team was merged into the Doubao system. Since then, there have been discussions internally about the similarities and differences between the button and "Doubao Work" products. Especially in the fiercely competitive situation of AI Agent office products from several major companies, resources are more inclined towards "tofu bun work".

Open the black box

Users cannot predict the cost of each task beforehand and cannot calculate it afterwards, feeling that using AI agents for token consumption is like opening a blind box.

At present, the billing standards for agents in the AI industry can be roughly divided into three schools:

One is the monthly quota system, represented by OpenAI's ChatGPT Plus, which is the most user-friendly model for novice users. By paying a fixed monthly subscription fee, users do not need to worry about bills, but there are quota restrictions, making it more suitable for personal assistant scenarios on the C-end. Once developers need to frequently call complex automated workflows, the quota system will become a bottleneck for productivity.

The second is a pure token billing system, using Dify, Alibaba Bailian DeepSeek、 Represented by developer platforms such as Zhipu. After a user executes a workflow, the log will clearly list how many tokens were consumed by the input, how many were consumed by the output, how many plugin calls were triggered, and even how many tokens were cached and hit, thus saving the user how much money.

The third is the point based packaging system. Most domestic AI Agent products adopt a point based packaging model, such as WorkBuddy, Doubao Work, Qianwen Office, etc. The platform does not disclose the conversion logic between points and token amounts, only annotating the corresponding multiple points consumption speed when calling different models.

Taking buttons as an example: In the points consumption details, users can only see the task type, dialogue content, and corresponding burned points, without knowing which steps the agent has executed and the corresponding points amount for each step. Therefore, it is difficult for users to shorten the agent's work chain and save points by optimizing prompt words.

In terms of transparency, WorkBuddy provides a relatively better solution. When the task begins to execute, an estimated point consumption interval will pop up in the dialogue, and the current consumed points will be displayed in real time with each step. If the point consumption exceeds expectations, the user can stop at any time and continue to execute after replacing the low-priced model.

The aforementioned developer's suggestion is that in the case of opaque point consumption, users can actively reduce unnecessary consumption based on their usage habits: firstly, try to open new sessions for different tasks to avoid infinite stacking of context in the same session, because the accumulated historical records of long sessions will be continuously reloaded, which will increase token consumption; Secondly, reduce the use of Auto automatic model routing and manually specify lightweight models for simple tasks to avoid being matched by the system to high priced long context models; The third is to refine high-frequency processes into skills, solidify commonly used steps, shorten single inference rounds, reduce duplicate searches and automatic retries.

(Author Liu Sixuan)