Migrating my family finance bot from Python to Rust (teloxide) [part 2]
Last updated on 21 December, 2022
In the first part I disassembled my desired results into several use cases:
- person wants to add check to certain category
- person wants to know how much money remain to spend
- money should be automatically added by specific schedule
- I should be notified if I save something in the end of the spending period
Also in the first part I've covered how did I built first command - /addcheck and all related actions around it. /addcheck allows user to choose category with a button, enter a check with free-form text and then remove messages so chat won't be polluted with them.
However, I have not covered my code with proper error-handling – there is almost no error handling at all, but we both (me and my reader) probably already know that error-handling is a strong side of Rust. I will be back to it once I validate all user scenarios and ensure that my concept works how I've imagined it myself in general.
So what I am doing here, in front of you, it's what I call 'prototyping': a bit dirty and fast work to achieve quick results.
My reader may judge me because of that, because I won't disclose to you how 'real-life' Rust projects work in production.
However, I plan to do public refactoring (dedicating another blog post for that) and pass code review with one (or few) of my fellow from Rust Ukraine Organisation afterwards. So stay tuned and let's continue our work together.
Second command: person wants to know how much money remain for spending
I'd create new file for the handler of our new command: src/remain.rs.
In the main.rs I'd add new command definition, like that:
...
enum Command {
#[command(description = "display this text.")]
Help,
#[command(description = "Додати витрати в певну категорію.")]
AddCheck,
#[command(description = "cancel the purchase procedure.")]
Cancel,
#[command(description = "Показати скільки грошей залишилося")]
Remain { category_name: String },
}
...
let command_handler = teloxide::filter_command::<Command, _>()
.branch(
case![State::Start]
.branch(case![Command::Help].endpoint(help))
.branch(case![Command::AddCheck].endpoint(start))
.branch(case![Command::Remain { category_name }].endpoint(remain_handler))
)
.branch(case![Command::Cancel].endpoint(cancel));
In the newly created remain.rs I'd describe my handler, like this:
...
pub async fn remain(
bot: Bot,
msg: Message,
api: NotionApi,
category_name: String,
) -> HandlerResult {
if let Ok(category) = Category::from_str(&category_name) {
let remain = get_how_much_money_remain(&api, category.to_string()).await;
if let Ok(remain) = remain {
bot.send_message(
msg.chat.id,
format!("По цій категорії залишилося {} євро", remain),
)
.await?;
};
} else {
bot.send_message(
msg.chat.id,
format!("Бот не зрозумів що це таке: {}", &category_name),
)
.await?;
}
Ok(())
}Let Ukrainian strings to not confuse you. The first one means: 'By this category {} euro remains' and the second one – 'Bot didn't understand this input: {}'.
We're handling two cases there: user entered the name of the category correctly and user entered incorrect string.
Reminder: don't forget to make get_how_much_money_remain in src/notion/mod.rs public so you'd be able to import it to the handler.
This is how it looks like in real life.
We do /remain {category name} and in case of success – we receive remaining amount.
Running Concurrent Futures
I acknowledge it's a hassle to enter a full category name yourself. To provide better user experience, I'd remove argument. It would be possible just to ack for /remain and have all categories answered at once.
My remain handler is fully changing.
...
pub async fn remain(bot: Bot, msg: Message, api: NotionApi) -> HandlerResult {
let categories = [
Category::FoodAndDrinks,
Category::Rent,
Category::UkrReponsibilities,
Category::OperationalSpends,
];
let cat_futures = categories
.iter()
.map(|category| get_how_much_money_remain(&api, category.clone()));
let message = join_all(cat_futures)
.await
.iter()
.filter_map(|res| res.as_ref().ok())
.map(|remain_amount| {
format!(
"По категорії '{}' залишилося {} євро.",
remain_amount.category.to_string(),
remain_amount.amount
)
})
.collect::<Vec<String>>()
.join("\n");
bot.send_message(msg.chat.id, message).await?;
Ok(())
}
...The most important work here does the function join_all. It comes from `futures' external crate.
...
futures = "0.3.25"
...I'd also make a struct which would help me to contain all information I need on remaining amount:
...
pub struct RemainAmount {
pub category: Category,
pub amount: f64,
}
...I'd slightly modify get_how_much_money_remain.
...
pub async fn get_how_much_money_remain(api: &NotionApi, category: Category) -> Result<RemainAmount, ()> {
let endpoint = format!("databases/{}/query", DATABASE_ID);
let kwargs = json!({
"filter": {
"property": "Tags",
"multi_select": {
"contains": category.to_string()
}
},
"sorts": [{
"property": "Date",
"timestamp": "created_time",
"direction": "descending"
}]
});
let assembled_url = format!("{}{}", NOTION_HOST, endpoint);
let response = api
.client
.post(assembled_url)
.header("Notion-Version", "2022-06-28")
.bearer_auth(&api.secret)
.json(&kwargs)
.send()
.await;
if let Ok(response) = response {
let json_resp = response.json::<Value>().await.unwrap();
let remain = json_resp.get("results").and_then(|value| {
value.get(0).and_then(|value| {
value.get("properties").and_then(|value| {
value.get("Remain").and_then(|value| {
value.get("formula").and_then(|value| value.get("number"))
})
})
})
});
if let Some(remain) = remain {
return Ok(RemainAmount {
category,
amount: remain.as_f64().unwrap(),
});
}
}
Err(())
}
...and make_new_notion_entry_for_check because type was changed.
...
pub async fn make_new_notion_entry_for_check(api: NotionApi, category: &Category, check: &Check) {
let remain = get_how_much_money_remain(&api, category.to_owned()).await;
if let Ok(remain) = remain {
let spending_document = SpendingDocument::new(
category.to_string(),
category,
check.spent,
remain.amount,
&check.entries,
);
post_new_notion_document(&api, spending_document).await;
}
}
...Also, don't forget to clean argument from Bot Command.
...
#[derive(BotCommands, Clone)]
#[command(
rename_rule = "lowercase",
description = "These commands are supported:"
)]
enum Command {
#[command(description = "display this text.")]
Help,
#[command(description = "Додати витрати в певну категорію.")]
AddCheck,
#[command(description = "cancel the purchase procedure.")]
Cancel,
#[command(description = "Показати скільки грошей залишилося")]
Remain,
}
...This is our result.
New project. Converting to Rust Workspace
The last few cases remain.
- money should be automatically added by specific schedule
- I should be notified if I save something in the end of the spending period
I thought it would be more convenient to separate worker which would do jobs on schedule, so I created this modules structure instead of the old one.
.
├── Cargo.lock
├── Cargo.toml
├── models
│ ├── Cargo.toml
│ └── src
│ └── lib.rs
├── notion
│ ├── Cargo.toml
│ └── src
│ ├── document.rs
│ ├── lib.rs
│ └── spending.rs
├── rust_bot
│ ├── Cargo.lock
│ ├── Cargo.toml
│ └── src
│ ├── add_check.rs
│ ├── main.rs
│ └── remain.rs
└── scheduler
├── Cargo.toml
└── src
└── main.rs
8 directories, 15 files
We'd have two libraries: notion and models, and two binary crates: bot and scheduler.
Dependency review
This is how I set up the workspace in the root Cargo.toml file.
[workspace]
members = ["rust_bot", "scheduler"]These are dependencies for my new scheduler.
[package]
name = "scheduler"
version = "0.1.0"
edition = "2021"
[dependencies]
async-std = "1.12.0"
chrono = "0.4.23"
job_scheduler = "1.2.1"
models = { path = "../models" }
reqwest = { version="0.11.13", features = ["blocking", "json"] }
notion = { path = "../notion" }
dotenv = "0.15.0"
pretty_env_logger = "0.4.0"These are my dependencies for notion library. It would store all my notion-related functions, structs and so on.
[package]
name = "notion"
version = "0.1.0"
edition = "2021"
[dependencies]
reqwest = {version="0.11.13", features=["json"]}
serde = {version="1.0.147", features = ["derive"]}
serde_json = "1.0.89"
models = { path = "../models" }
chrono = "0.4.23"And dependencies for models, which is just regex. You'd see – I kept my models as simple as possible.
[package]
name = "models"
version = "0.1.0"
edition = "2021"
[dependencies]
regex = "1.7.0"
Inside the models library
Every entity you may remember from the previous article, would now be transferred to models.
Those are: Category, Check, CheckEntry, RemainAmount. Those structs are basis for my applications. They model my real-life entities which I operate with.
In the models crate, there are also trait implementations for those structs: FromStr, ToString, methods and couple of new:
- ToSchedule with the function
replenish_money_schedules. It is responsible for converting myCategoryto array of schedules, formatted as cron strings.
...
impl ToSchedule for Category {
fn replenish_money_schedules(&self) -> Vec<String> {
match self {
// https://crontab.cronhub.io/
Category::FoodAndDrinks | Category::OperationalSpends => vec![
String::from_str("0 0 0 15 * *").unwrap(),
String::from_str("0 0 0 15 * *").unwrap(),
],
Category::Rent | Category::UkrReponsibilities => {
vec![String::from_str("0 0 0 1 * *").unwrap()]
}
}
}
}
...- ToMoneyReplenishment with the function
to_replenishment_amount. It would convert myCategoryto amount of money which would be added by schedule. Each time period I'm adding the same amount of money to each category.
...
impl ToMoneyReplenishment for Category {
fn to_replenishment_amount(&self) -> f64 {
match self {
Category::FoodAndDrinks => 155.0,
Category::OperationalSpends => 250.0,
Category::UkrReponsibilities => 310.0,
Category::Rent => 1140.0,
}
}
}
...(not sure this naming is clear, thought)
Alltogether it looks like that:
use std::str::FromStr;
use regex::Regex;
#[derive(Debug, Clone, Copy)]
pub enum Category {
FoodAndDrinks,
OperationalSpends,
Rent,
UkrReponsibilities,
}
impl ToString for Category {
fn to_string(&self) -> String {
let string_literal = match self {
Category::FoodAndDrinks => "Поживні речовини",
Category::OperationalSpends => "Операційні витрати",
Category::Rent => "Аренда",
Category::UkrReponsibilities => "Українські зобовʼязання",
};
string_literal.to_owned()
}
}
impl FromStr for Category {
type Err = ();
fn from_str(s: &str) -> Result<Self, Self::Err> {
match s {
"Поживні речовини" => Ok(Category::FoodAndDrinks),
"Операційні витрати" => Ok(Category::OperationalSpends),
"Аренда" => Ok(Category::Rent),
"Українські зобовʼязання" => Ok(Category::UkrReponsibilities),
_ => Err(()),
}
}
}
impl Category {
pub fn to_storage_id(&self) -> &str {
match self {
Category::FoodAndDrinks => "de702077-821a-4b59-9ab0-949ca954c4a6",
Category::OperationalSpends => "3e392471-bd4a-4cde-befb-e52e8f31b26d",
Category::Rent => "278a679f-cde4-44aa-be5a-2876b67d080b",
Category::UkrReponsibilities => "M?[|",
}
}
}
#[derive(Debug)]
pub struct Check {
pub entries: Vec<CheckEntry>,
pub spent: f64,
}
#[derive(Debug)]
pub struct CheckEntry(String, f64);
impl FromStr for Check {
type Err = ();
fn from_str(s: &str) -> Result<Self, Self::Err> {
let mut check = Check {
entries: s
.lines()
.map(|line| CheckEntry::from_str(line).expect("Line is corrupted"))
.collect::<Vec<CheckEntry>>(),
spent: 0.0,
};
check.spent = check.entries.iter().map(|entry| entry.1).sum::<f64>();
Ok(check)
}
}
impl FromStr for CheckEntry {
type Err = ();
fn from_str(s: &str) -> Result<Self, Self::Err> {
let content = s
.split_ascii_whitespace()
.into_iter()
.collect::<Vec<&str>>();
let re = Regex::new(r"[-+]?\d*\.\d+|\d+").unwrap();
let mat = re.find(content.last().unwrap());
if let Some(matched) = mat {
let price = matched.as_str().parse::<f64>().unwrap();
let description = content
.iter()
.take(content.len() - 1)
.map(|el| el.to_string())
.collect::<Vec<String>>()
.join(" ");
return Ok(CheckEntry(description, price));
}
Err(())
}
}
impl ToString for CheckEntry {
fn to_string(&self) -> String {
format!("{} — ціна: {} (в євро)", self.0, self.1)
}
}
pub struct RemainAmount {
pub category: Category,
pub amount: f64,
}
pub trait ToSchedule {
fn replenish_money_schedules(&self) -> Vec<String>;
}
impl ToSchedule for Category {
fn replenish_money_schedules(&self) -> Vec<String> {
match self {
// https://crontab.cronhub.io/
Category::FoodAndDrinks | Category::OperationalSpends => vec![
String::from_str("0 0 0 15 * *").unwrap(),
String::from_str("0 0 0 15 * *").unwrap(),
],
Category::Rent | Category::UkrReponsibilities => {
vec![String::from_str("0 0 0 1 * *").unwrap()]
}
}
}
}
pub trait ToMoneyReplenishment {
fn to_replenishment_amount(&self) -> f64;
}
impl ToMoneyReplenishment for Category {
fn to_replenishment_amount(&self) -> f64 {
match self {
Category::FoodAndDrinks => 155.0,
Category::OperationalSpends => 250.0,
Category::UkrReponsibilities => 310.0,
Category::Rent => 1140.0,
}
}
}Inside the notion library. Builder pattern.
.
├── Cargo.toml
└── src
├── document.rs
├── lib.rs
└── spending.rs
1 directory, 4 files
I used to have spending_document.rs and mod.rs. Now the code is slightly refactored, so both binaries: bot and scheduler – would be able to benefit from using it.
use chrono::Local;
use models::Category;
use serde::Serialize;
use serde_json::json;
use serde_json::Value;
use crate::spending::SpendingDocumentBulletPoints;
#[derive(Serialize)]
pub struct Document {
parent: Value,
icon: Value,
properties: Value,
children: Vec<Value>,
}
impl Document {
pub fn new(database_id: String) -> Document {
Document {
parent: json!({ "database_id": database_id }),
properties: json!(null),
children: Vec::new(),
icon: json!(null),
}
}
pub fn convert_to_spending<T: ToString>(
self,
title: String,
category: &Category,
spent: f64,
remain: f64,
entries: &Vec<T>,
) -> Document {
let properties = json!({
"Name": {
"type": "title",
"title": [
{
"plain_text": title,
"text": {
"content": title,
"link": serde_json::Value::Null},
"type": "text"
}
]
},
"Tags": {
"multi_select": [{
"id": category.to_storage_id(),
}],
"type": "multi_select"
},
"Date": {"date": {"start": Local::now().to_rfc3339()}, "type": "date"},
"Spent": {"number": -spent, "type": "number"},
"For current period": {"number": remain, "type": "number"},
});
let bullets = SpendingDocumentBulletPoints::new(entries);
Document { parent: self.parent, icon: self.icon, properties, children: bullets.bullets }
}
pub fn convert_to_replenishment(
self,
category: &Category,
amount_to_add: f64,
) -> Document {
let assembled_title = format!("{} - бот відкриває період", category.to_string());
let properties = json!({
"Name": {
"type": "title",
"title": [
{
"plain_text": assembled_title,
"text": {
"content": assembled_title,
"link": serde_json::Value::Null},
"type": "text"
}
]
},
"Tags": {
"multi_select": [{
"id": category.to_storage_id(),
}],
"type": "multi_select"
},
"Date": {"date": {"start": Local::now().to_rfc3339()}, "type": "date"},
"Spent": {"number": 0, "type": "number"},
"For current period": {"number": amount_to_add, "type": "number"},
});
let icon = json!({"emoji": "💸"});
Document { parent: self.parent, icon, properties, children: self.children }
}
}As I'm having several types of documents I want to create – I made chaining constructors for them. It looks like that:
...
Document::new(DATABASE_ID.to_string()).convert_to_spending(
category.to_string(),
category,
check.spent,
remain.amount,
&check.entries,
);
...To be honest: to_string there is not necessary, as I can tell Rust that it can accept references to static strings.
use serde_json::json;
use serde_json::Value;
pub struct SpendingDocumentBulletPoints {
pub bullets: Vec<Value>,
}
impl SpendingDocumentBulletPoints {
pub fn new<T: ToString>(entries: &Vec<T>) -> Self {
SpendingDocumentBulletPoints {
bullets: entries
.into_iter()
.map(|line| {
json!({
"object": "block",
"type": "bulleted_list_item",
"bulleted_list_item":
{
"rich_text": [{
"type": "text",
"text": {
"content": line.to_string(),
}
}]
}
})
})
.collect::<Vec<Value>>(),
}
}
}Spending.rs doesn't have much. That code can be also generalised and refactored, but I decided to do it afterwards.
mod spending;
pub mod document;
use models::Category;
use models::Check;
use models::RemainAmount;
use reqwest::Client;
use serde::Serialize;
use serde_json::json;
use serde_json::Value;
use crate::document::Document;
pub const DATABASE_ID: &str = "<redacted>";
const NOTION_HOST: &str = "https://api.notion.com/v1/";
#[derive(Debug, Clone)]
pub struct NotionApi {
secret: String,
client: Client,
}
impl NotionApi {
pub fn from_env() -> Self {
NotionApi {
secret: std::env::var("NOTION_SECRET").unwrap(),
client: Client::new(),
}
}
}
pub async fn make_new_notion_entry_for_check(api: NotionApi, category: &Category, check: &Check) {
let remain = get_how_much_money_remain(&api, category.to_owned()).await;
if let Ok(remain) = remain {
let spending_document: Document = Document::new(DATABASE_ID.to_string()).convert_to_spending(
category.to_string(),
category,
check.spent,
remain.amount,
&check.entries,
);
post_new_notion_document::<Document>(&api, spending_document).await;
}
}
pub async fn post_new_notion_document<T: Serialize>(api: &NotionApi, document: T) {
let endpoint = "pages";
let assembled_url = format!("{}{}", NOTION_HOST, endpoint);
let response = api
.client
.post(assembled_url)
.header("Notion-Version", "2022-06-28")
.bearer_auth(&api.secret)
.json(&document)
.send()
.await;
if let Ok(response) = response {
let _json_resp = response.json::<Value>().await.unwrap();
println!("posted");
}
}
pub async fn get_how_much_money_remain(api: &NotionApi, category: Category) -> Result<RemainAmount, ()> {
let endpoint = format!("databases/{}/query", DATABASE_ID);
let kwargs = json!({
"filter": {
"property": "Tags",
"multi_select": {
"contains": category.to_string()
}
},
"sorts": [{
"property": "Date",
"timestamp": "created_time",
"direction": "descending"
}]
});
let assembled_url = format!("{}{}", NOTION_HOST, endpoint);
let response = api
.client
.post(assembled_url)
.header("Notion-Version", "2022-06-28")
.bearer_auth(&api.secret)
.json(&kwargs)
.send()
.await;
if let Ok(response) = response {
let json_resp = response.json::<Value>().await.unwrap();
let remain = json_resp.get("results").and_then(|value| {
value.get(0).and_then(|value| {
value.get("properties").and_then(|value| {
value.get("Remain").and_then(|value| {
value.get("formula").and_then(|value| value.get("number"))
})
})
})
});
if let Some(remain) = remain {
return Ok(RemainAmount {
category,
amount: remain.as_f64().unwrap(),
});
}
}
Err(())
}
Lib.rs remains almost the same as mod.rs was.
Scheduler. Running async inside the sync main. Move values into closures, so jobs would be the owners
.
├── Cargo.toml
└── src
└── main.rs
1 directory, 2 files
Scheduler is a very simple and small binary package. It's just a runner for cron jobs.
use async_std::task;
use job_scheduler::Job;
use job_scheduler::JobScheduler;
use models::Category;
use models::ToMoneyReplenishment;
use models::ToSchedule;
use notion::document::Document;
use notion::post_new_notion_document;
use notion::NotionApi;
use notion::DATABASE_ID;
fn main() {
dotenv::dotenv().ok();
pretty_env_logger::init();
let mut sched = JobScheduler::new();
let categories = [
Category::FoodAndDrinks,
Category::Rent,
Category::UkrReponsibilities,
Category::OperationalSpends,
];
let mut jobs = Vec::new();
categories.into_iter().for_each(|category| {
let schedules = category.replenish_money_schedules();
schedules.into_iter().for_each(|schedule| {
let job = Job::new(schedule.parse().unwrap(), move || {
log::info!("Executing the job for {}", category.to_string());
let replenishment_document = Document::new(DATABASE_ID.to_owned())
.convert_to_replenishment(&category, category.to_replenishment_amount());
task::block_on(post_new_notion_document(
&NotionApi::from_env(),
replenishment_document,
));
});
jobs.push(job);
});
});
for job in jobs.into_iter() {
sched.add(job);
}
loop {
sched.tick();
std::thread::sleep(sched.time_till_next_job());
}
}
It uses async_std to run futures and models to have all data its need for each category.
Bot binary becomes thick as we've moved out everything except handlers
.
├── Cargo.lock
├── Cargo.toml
└── src
├── add_check.rs
├── main.rs
└── remain.rs
1 directory, 5 files
All handlers remain the same. I don't make any changes there.
Deployment to Google Cloud Run
For building and pushing image to container registry I used this workflow, combining with native GoogleCloud auth action.
Google Cloud did a good post, explaining how it works under the hood and how to set it up.
FROM rust:latest AS builder
LABEL service=universal
COPY . .
RUN apt-get update && apt-get install -y libclang-dev
RUN cargo build --release
FROM ubuntu
COPY ./target/release/scheduler ./scheduler
COPY ./target/release/telegram-bot ./telegram-bot
RUN apt-get update && apt-get install -y wget
RUN wget http://nz2.archive.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.16_amd64.deb
RUN dpkg -i libssl1.1_1.1.1f-1ubuntu2.16_amd64.deb
RUN dpkg --search libssl | grep libssl.so.1.1
RUN ldconfig -p | grep 'libssl'
ENTRYPOINT [ "./telegram-bot" ]
Be careful with 'libssl' stuff. Seems like telegram bot rely on some old dependency and it relies on 'libssl.so.1.1'.
This...
...
RUN apt-get update && apt-get install -y wget
RUN wget http://nz2.archive.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.16_amd64.deb
RUN dpkg -i libssl1.1_1.1.1f-1ubuntu2.16_amd64.deb
RUN dpkg --search libssl | grep libssl.so.1.1
RUN ldconfig -p | grep 'libssl'
......is my workaround to make it work.
To grab a link to libssl version, you may use 'Index of /ubuntu/pool/main/o/openssl'.
target
*/target/*
.env
.githubDon't forget to exclude target from your context. Otherwise, you'd have your build context grew to mammoth sizes.
name: deploy
on:
push:
branches:
- 'main'
jobs:
build-and-push-to-gcr:
runs-on: ubuntu-latest
permissions:
id-token: write # This is required for requesting the JWT
contents: read # This is required for actions/checkout
steps:
- uses: actions/checkout@v3
- name: Authenticate to Google Cloud
id: auth
uses: google-github-actions/auth@v1.0.0
with:
create_credentials_file: 'true'
workload_identity_provider: <redacted>
service_account: <redacted>
- uses: RafikFarhad/push-to-gcr-github-action@v5-beta
with:
registry: gcr.io
project_id: <redacted>
image_name: bot_project_rust
image_tag: universalThese two steps pushing my freshly-built docker images with different binaries to my Google Cloud Container Registry.
After I had my image in place, I've manually deployed service in Google Cloud Run.
It was marked as unhealthy. But it works.
There are some leftovers, which I would cover in the third part.
- (commit reference: 1)
- (commit reference: 2)
- (commit reference: 3)
- (commit reference: 4)
- (commit reference: 5)
- (commit reference: 6)
might be squashed later.
You can subscribe on my newsletters
Let's see if we can become internet friends.