The Cost of a Session

Optional, for readers who know a little Python.

In Paying by the Token you saw that every turn of a conversation sends the whole history again, and in The Same Start Every Time that caching makes the repeated part cheap. Here you work out how much difference that makes.

Write the function session_cost(price, new_tokens). It gets:

• price: the price in dollars per million input tokens, like 4.0
• new_tokens: a list of how many tokens each turn adds, oldest first, like [30000, 2000, 2000]

Use this simple model of a session:

1. Each turn sends everything sent on the turn before, plus its own new tokens. With [30000, 2000, 2000], the turns send 30,000, then 32,000, then 34,000 tokens: 96,000 input tokens in all.
2. Without caching, every input token costs the full price: 96,000 tokens at 4 dollars per million is 0.3840 dollars.
3. With caching, everything sent on the turn before is read from the cache at a tenth of the price, and only the turn's new tokens are paid in full. The first turn has nothing cached, so it is all full price. Turn 2 here pays 30,000 tokens at 0.4 dollars per million and 2,000 at 4 dollars per million.

Return the total number of input tokens, the cost without caching and the cost with caching. The code at the bottom prints them.

Example input (the price, then each turn's new tokens):

4
30000 2000 2000

Output:

turns: 3
input tokens sent: 96000
without caching: 0.3840 dollars
with caching: 0.1608 dollars

This model leaves out three things real caching has: writing to the cache costs a little more than normal input, the cache expires after a few minutes without use, and output tokens are paid separately. None of them changes the big picture. Once you've passed, try the 40-turn session in the tests and compare the two costs.