Bots and backends
Telegram bots and server-side sources — HMAC instead of Telegram IDs, server-side sessions, and context.ip.
A bot or a backend has no screen and no foreground, so your server has to work out identity, sessions, and geo on its own. This recipe shows how to do it in Go without third-party dependencies. The logic is the same in other languages.
A bot uses the bot platform, a backend uses server. If the bot or backend serves the same people
as a mobile app, add the platform to that app and send events with its key: a person with the same
user_id in the bot and in the app is counted once. For a standalone bot or service, create a
separate app. Learn more in Event platform.
User identity
A Telegram ID is someone else's identifier, and a public one: it makes a person easy to find. It
shouldn't end up in analytics as is. The server doesn't reject such strings (purely numeric
user_id values are allowed), so protecting them is the bot's job:
user_id= HMAC-SHA256 of the Telegram ID keyed with the bot's secret, in hex (64 characters). Without the secret, the Telegram ID can't be recovered from it, and the same person always gets the sameuser_id.install_id— a stable UUID per chat. It's convenient to derive it from the same HMAC: then you don't need to store a mapping table.
Store the secret like the bot's other secrets, for example in the ANALYTICS_HMAC_SECRET variable.
If you change the secret, every user becomes a new one.
package main
import (
"crypto/hmac"
"crypto/sha256"
"encoding/hex"
"fmt"
"os"
"strconv"
)
// UserID returns the analytics user_id: the HMAC-SHA256 of the user's Telegram ID, in hex.
func UserID(secret []byte, telegramUserID int64) string {
mac := hmac.New(sha256.New, secret)
mac.Write([]byte("user:" + strconv.FormatInt(telegramUserID, 10)))
return hex.EncodeToString(mac.Sum(nil))
}
// InstallID returns a stable install_id for the chat: the first 16 HMAC bytes as a UUIDv8 (RFC 9562).
func InstallID(secret []byte, chatID int64) string {
mac := hmac.New(sha256.New, secret)
mac.Write([]byte("chat:" + strconv.FormatInt(chatID, 10)))
b := mac.Sum(nil)[:16]
b[6] = (b[6] & 0x0f) | 0x80
b[8] = (b[8] & 0x3f) | 0x80
return fmt.Sprintf("%x-%x-%x-%x-%x", b[0:4], b[4:6], b[6:8], b[8:10], b[10:16])
}
func main() {
secret := []byte(os.Getenv("ANALYTICS_HMAC_SECRET"))
fmt.Println("user_id: ", UserID(secret, 123456789))
fmt.Println("install_id:", InstallID(secret, 123456789))
}In a private chat, chat_id equals the user's ID, so each person has a single "install". In groups,
every chat is an install of its own, and a person is stitched together by user_id (see
Identity).
Server-side sessions
A bot session is a series of messages from a user with no gap longer than 30 minutes:
- The first message after a gap opens a session:
session_startwith that message's time. - Every following message extends the session.
- If the user stays silent for 30 minutes, the session closes:
session_endwith the time of the last message andduration_ms= last message − first message.
timestamp is the time of the event itself, not the moment the batch is sent. A session with a
single message has zero duration — that's fine, the session still makes it into reports.
When the bot shuts down, close the open sessions and flush the queue. Otherwise the session_end
events are lost, and the average session duration comes out too low.
Queue and sending
Sending an event for every message is wasteful. The queue accumulates events and sends them as a batch every 5 seconds or as soon as 500 events pile up. If ingest is unavailable, the batch stays in the queue until the next attempt. The full retry policy is in Errors and retries and in the client from the Go recipe.
package main
import (
"bytes"
"context"
crand "crypto/rand"
"encoding/json"
"fmt"
"log"
"net/http"
"os"
"os/signal"
"sync"
"time"
)
const (
endpoint = "https://i.analytics.example.com/v1/events"
idleTimeout = 30 * time.Minute
flushEvery = 5 * time.Second
maxBatch = 500
)
type Event struct {
InsertID string `json:"insert_id"`
Name string `json:"name"`
Timestamp time.Time `json:"timestamp"`
InstallID string `json:"install_id"`
SessionID string `json:"session_id"`
UserID string `json:"user_id,omitempty"`
DurationMs *int64 `json:"duration_ms,omitempty"`
}
type session struct {
id, installID, userID string
start, last time.Time
}
// Tracker keeps track of the bot users' sessions and accumulates events for sending.
type Tracker struct {
key string
http *http.Client
mu sync.Mutex
sessions map[string]*session // keyed by install_id
queue []Event
}
func NewTracker(key string) *Tracker {
return &Tracker{key: key, http: &http.Client{Timeout: 10 * time.Second}, sessions: map[string]*session{}}
}
// Touch records a user message received at time at.
func (t *Tracker) Touch(installID, userID string, at time.Time) {
t.mu.Lock()
defer t.mu.Unlock()
s := t.sessions[installID]
if s != nil && at.Sub(s.last) >= idleTimeout {
t.end(s)
s = nil
}
if s == nil {
s = &session{id: newUUID(), installID: installID, userID: userID, start: at, last: at}
t.sessions[installID] = s
t.queue = append(t.queue, Event{
InsertID: newUUID(), Name: "session_start", Timestamp: at,
InstallID: installID, SessionID: s.id, UserID: userID,
})
return
}
s.last = at
}
// CloseIdle closes sessions idle longer than idleTimeout; all closes every session (bot shutdown).
func (t *Tracker) CloseIdle(now time.Time, all bool) {
t.mu.Lock()
defer t.mu.Unlock()
for _, s := range t.sessions {
if all || now.Sub(s.last) >= idleTimeout {
t.end(s)
}
}
}
func (t *Tracker) end(s *session) {
duration := s.last.Sub(s.start).Milliseconds()
t.queue = append(t.queue, Event{
InsertID: newUUID(), Name: "session_end", Timestamp: s.last,
InstallID: s.installID, SessionID: s.id, UserID: s.userID, DurationMs: &duration,
})
delete(t.sessions, s.installID)
}
// Flush sends the queue in batches. On error, unsent events go back to the front of the queue.
func (t *Tracker) Flush(ctx context.Context) error {
t.mu.Lock()
pending := t.queue
t.queue = nil
t.mu.Unlock()
for len(pending) > 0 {
n := min(len(pending), maxBatch)
if err := t.send(ctx, pending[:n]); err != nil {
t.mu.Lock()
t.queue = append(pending, t.queue...)
t.mu.Unlock()
return err
}
pending = pending[n:]
}
return nil
}
func (t *Tracker) send(ctx context.Context, events []Event) error {
body, err := json.Marshal(map[string]any{
"sent_at": time.Now(),
// The bot doesn't know the user's IP: "none" disables geo instead of locating the bot's server.
"context": map[string]string{"platform": "bot", "app_version": "1.0.0", "ip": "none"},
"events": events,
})
if err != nil {
return fmt.Errorf("marshal batch: %w", err)
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, endpoint, bytes.NewReader(body))
if err != nil {
return fmt.Errorf("new request: %w", err)
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("X-App-Key", t.key)
resp, err := t.http.Do(req)
if err != nil {
return fmt.Errorf("send batch: %w", err)
}
defer resp.Body.Close()
switch {
case resp.StatusCode == http.StatusOK:
return nil
case resp.StatusCode == http.StatusTooManyRequests || resp.StatusCode >= 500:
return fmt.Errorf("ingest: HTTP %d, retry later", resp.StatusCode)
default:
// 4xx: a retry won't fix the batch, so log it and don't put it back in the queue.
log.Printf("ingest: HTTP %d, batch dropped", resp.StatusCode)
return nil
}
}
func newUUID() string {
var b [16]byte
_, _ = crand.Read(b[:])
b[6] = (b[6] & 0x0f) | 0x40
b[8] = (b[8] & 0x3f) | 0x80
return fmt.Sprintf("%x-%x-%x-%x-%x", b[0:4], b[4:6], b[6:8], b[8:10], b[10:16])
}
func main() {
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt)
defer stop()
tracker := NewTracker(os.Getenv("AM_PK"))
// In a real bot, Touch is called from the handler of every incoming message.
tracker.Touch("0f8fad5b-d9cb-469f-a165-70867728950e", "", time.Now())
ticker := time.NewTicker(flushEvery)
defer ticker.Stop()
for {
select {
case <-ticker.C:
tracker.CloseIdle(time.Now(), false)
if err := tracker.Flush(ctx); err != nil {
log.Print(err)
}
case <-ctx.Done():
tracker.CloseIdle(time.Now(), true)
flushCtx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
if err := tracker.Flush(flushCtx); err != nil {
log.Print(err)
}
cancel()
return
}
}
}The queue in this example lives in memory: if the process crashes, the events are lost. If that
matters, keep the queue in Redis or in a table in the bot's database. Delete events older than 7 days
before sending — the server would reject them anyway with the event_too_old code.
context.ip
Country and city are determined from the IP address. By default, that's the address the HTTP request came from. That's correct for mobile apps and websites, but not for server-side sources: the request comes from your server's address.
| Source | What to pass in context.ip |
|---|---|
| App, website | Nothing: the server uses the client's address |
| Telegram bot | "none": Telegram doesn't share the user's IP |
| Backend that knows the user's IP | The user's IP from the request to your backend |
| Cron job, internal service | "none" |
The IP is used only to look up the country and city and is discarded right away: it isn't stored in
ClickHouse, in logs, or in rejected events. An invalid address doesn't get the event rejected: the
event is accepted with the geo_ip_invalid warning and without geo.
If a batch contains events from different users, pass the IP in each event's context — it
overrides the batch context field by field:
package main
import (
"crypto/rand"
"fmt"
"log"
"net"
"net/http"
"strings"
"time"
)
type EventContext struct {
IP string `json:"ip,omitempty"`
}
type Event struct {
InsertID string `json:"insert_id"`
Name string `json:"name"`
Timestamp time.Time `json:"timestamp"`
InstallID string `json:"install_id"`
SessionID string `json:"session_id"`
Context *EventContext `json:"context,omitempty"`
}
// userIP returns the user's address behind your load balancer: the last X-Forwarded-For address,
// the one appended by your trusted proxy. Without a proxy, it's the address of the TCP connection.
func userIP(r *http.Request) string {
if xff := r.Header.Get("X-Forwarded-For"); xff != "" {
parts := strings.Split(xff, ",")
return strings.TrimSpace(parts[len(parts)-1])
}
host, _, err := net.SplitHostPort(r.RemoteAddr)
if err != nil {
return "none"
}
return host
}
// pending is read by the batch sender — one like Tracker.Flush in sessions.go.
var pending = make(chan Event, 1024)
func main() {
http.HandleFunc("POST /login", func(w http.ResponseWriter, r *http.Request) {
event := Event{
InsertID: newUUID(),
Name: "session_start",
Timestamp: time.Now(),
InstallID: r.Header.Get("X-Install-Id"),
SessionID: newUUID(),
Context: &EventContext{IP: userIP(r)},
}
select {
case pending <- event:
default: // the queue is full: the user's response doesn't wait for analytics
}
w.WriteHeader(http.StatusNoContent)
})
go func() {
for range pending { // the batch sender runs here
}
}()
log.Fatal(http.ListenAndServe(":8080", nil))
}
func newUUID() string {
var b [16]byte
_, _ = rand.Read(b[:])
b[6] = (b[6] & 0x0f) | 0x40
b[8] = (b[8] & 0x3f) | 0x80
return fmt.Sprintf("%x-%x-%x-%x-%x", b[0:4], b[4:6], b[6:8], b[8:10], b[10:16])
}Verification
Send a single event from the terminal, as in the curl recipe, and open the app's Integration tab: the event shows up within a few seconds. Rejected events are listed there too, with their reason codes.