Yaba-2.0 Architecture

Yaba-2.0 is still in the works. This will come with an overhaul of the platform. I will still maintain yaba.markizano.net as a public interface to the app for you to be able to "try it before you buy it". However, I am going to expand on how the architecture will lay out with regards to a self-managed vs a hosted version of this application. Ideas keep coming on how I can manage this and I am so excited to share my designs with you and get your feedback on the direction being taken. This is still something we can construct together and build a team behind the project if it becomes useful.

Yaba Recap

If you're completely new to the project, welcome! Yaba is Yet Another Budgeting App. I got sick and tired of downloading CSV files from all my banks and managing them in complex Excel books just to be able to visualize my data around my spending so I can budget more efficiently. I built Yaba as a means of organizing my transactions in a way that's different than most apps. Most apps want you to treat a transaction like a bar you drop in a bucket and you can't do anything else with it. Whereas I want to sometimes attach a color coded sticky note to my transaction and then put on different color filtering lenses on to see my spending under different lights. Yaba let's you do this! Also, I don't like the idea of selling people's information without their knowledge or consent, so this app will always provide a version you can download and use to your hearts content whilst having the Ethernet cable (or Wi-Fi) disabled and unplugged! I want you to have the power of your data in your hands and make it as easy and streamlined as possible to manage your budgets and make it easy to develop healthy spending habits (you know, the ones you are aware of!).

This started as a PHP script a long time ago where I would munch together a bunch of CSV files together from my banks to create readable tables of how much I was spending where in my terminal. It's since evolved into a web platform that can serve as both an isolated app (like Word 2007 before Microsoft made it subscription based) or as a hosted platform with access to streaming your data using third party services like Stripe to connect your accounts and keep your data in sync. I am still building the hosted part out in this rewrite. More to come in this post now!

AngularJS -> Angular 2 Overhaul

This project has been completely ported from the since deprecated AngularJS to the TypeScript version of Angular 2 (actually, it's version 17 as of the writing of this blog, but you get what I mean, right?). So I have gone through and re-done just about every component and working to ensure its still as functional as a desktop-app while building the hosted aspect as well.

ElectronJS Addition

In order to have this working on all platforms, I chose ElectronJS to house the Yaba application. In this way, it won't matter if you are on Windows, OSX or Linux -- you can use Yaba! I plan on making the web platform mobile-first designs so use on iOS and Android are as seamless as can be as well. When I go to package the app, I can create different types of deployments depending on the target being catered.

Yaba Architecture

I have a pretty architecture picture to share with you 😍

1) Users will be approaching the application with their CSV and XLS documents. I want to be able to parse them all and injest any and all statement data from any bank (eventually). For now, Yaba only supports CSV files. It parses the headers and interprets where to map them for its canonical model of a transaction.

2) Yaba will injest these CSV and XLS files and munge the data for us to crunch the numbers so we don't have to. Eventually, I'd like to get it to the point where it can sync with services like Plaid and Stripe to get your transaction history for banks that support those channels. I'm sure they aren't free pipe access, so this is where hosting costs, security compliance laws and all that jazz will come into play, thus sparking the need for a subscription fee if you will be using the hosted version for the streamlined access to all your transaction history and spend/income in one place.

3) Data can be stored either locally in a simple sqlite3 database or via a MongoDB if using the hosted version. I plan on ensuring Yaba supports microservices and HA and all those other fun things sysadmins must deal with on a daily basis. I know I can setup adapters and driver patterns against the databases to ensure consistent storage mechanisms no matter the storage choice in the software itself.

4) The results will then be rendered to users in table, chart, graph or other reports the user can then use for other purposes. I would love for this to eventually grow into a tax-document production software one can use open-source and free rather than having to pay for services like TurboTax to complete your taxes. I would clearly disclaim that this software is merely a tool for this and cannot guarantee accuracy according to the laws, but we can collectively come together to ensure this software stays up to date with IRS regulations in producing for you the best tax return for you!

@FutureFeature

Other features I want to include in this as I continue developing:

• Profile Concept: I have different books I need to manage. If they are my own personal finances, my FinTech company finances, or my LMT finances -- I need a way to keep these segregated by switching "profiles".
• Personal or Business: Both need to track finances, but businesses have more to consider. Eventually, as I learn about business financing, I want to include support for features into this software as we continue too grow this out.

Feedback

I'm curious about what you think about this so far. Do you know of other solutions that might serve these purposes better? In what ways? I'm curious to know what you think -- drop a comment below and we can find out more in detail!

ytffmpeg-1.1.0 released!

The wait is finally over! ytffmpeg-1.1.0 is now available! This comes with a few new features and some tested workflows that streamline activities around video creation, subtitle generation and now with the ability to post the results to an SFTP endpoint!

Project Page

https://ytffmpeg.markizano.net/

Download

Download from Github

What is ytffmpeg?

ytffmpeg is a python project that was put together to automate workflows when it comes to video editing, subtitle generation and video composition. I refuse to pay for software on a license like Adobe Illustrator. Most of the awesome video editors out there were made specifically for a Mac or Windows and have little to no support for Linux. OpenShot-QT crashes too consistently to be used reliably. Also, personally, I find the GUI-version of video editing to be quite distasteful. It just doesn't seem like I can get the level of precision on a frame-by-frame basis like I can get with ffmpeg. Being able to drill down to the very frame of the video and make perfect cuts every time rather than a slider (try editing a 10-min video in-app on TikTok -- big fingers, tiny buttons). Kdenlive became slow and would crash when editing videos >~8mins...

Also, with my background in CloudOps/DevOps, I became accustomed to the idea of using configuration files to describe the final result.

ytffmpeg was designed to be simple enough to get started and making videos quickly, but also allow for the depth and complexity one may desire or need from more advanced operations. By having the full power of ffmpeg at your fingertips, but not necessarily requiring to learn all of the filters available in ffmpeg, let alone the chain of filters you sometimes have to put together because you changed one property of a video snippet. For example, having to reset the presentation timestamp of a snippet before concatenating a few streams together, or resetting the sample aspect ratio after scaling or cropping content. If that sounds complicated, don't worry -- ytffmpeg will take the rough edges off for you in due time with its simple configuration, I plan to make data structures take the place of the complex filter graphs to help you better visualize the results of your project.

Install your own DNS Server

I want to do a series on "rolling your own" and how you can use open-source software for just about all your needs. In this way, you can learn about how some software architectures work under the hood, how to configure, how to research and understand what is needed in order to get something like this setup and running. If you're like me, you're probably on a time-budget, so I won't delay, let's get right into it.

Install bind9

Let's start with the software required. If you are starting from a bare headless linux install like a Docker container or a fresh image, there may be a few things to install like your favorite text editor since we will be editing files. If you don't know of a text editor in Linux, you can use nano (yes, I probably just infuriated a third of you -- I can already see you in the comments section). I like it because it's simple and key-combination based, rather than arcane spell based as far as the navigation is concerned, but remember: it's all about preference and what works for you.

You can install bind9 with the following command (as of Debian Bookworm / Devuan Daedalus):

sudo apt-get install bind9 bind9-host bind9-utils bind9-dnsutils

Configuration

Once you have the service installed, it will be ready for you to configure. Let's checkout /etc/bind/:

$ tree -Ca /etc/bind 
/etc/bind
|-- bind.keys
|-- db.0
|-- db.127
|-- db.255
|-- db.empty
|-- db.local
|-- named.conf
|-- named.conf.default-zones
|-- named.conf.local
|-- named.conf.options
|-- rndc.key
`-- zones.rfc1918

0 directories, 12 files

In this directory, we see the configuration for named (I pronounce it "name-dee") which is the underlying name of the service. There's the root configuration, which is just an include statement to the other configuration files as named.conf.options, named.conf.local and named.conf.default-zones. Let's checkout named.conf.options:

$ cat /etc/bind/named.conf.options 
options {
	directory "/var/cache/bind";

	// If there is a firewall between you and nameservers you want
	// to talk to, you may need to fix the firewall to allow multiple
	// ports to talk.  See http://www.kb.cert.org/vuls/id/800113

	// If your ISP provided one or more IP addresses for stable 
	// nameservers, you probably want to use them as forwarders.  
	// Uncomment the following block, and insert the addresses replacing 
	// the all-0's placeholder.

	// forwarders {
	// 	0.0.0.0;
	// };

	//========================================================================
	// If BIND logs error messages about the root key being expired,
	// you will need to update your keys.  See https://www.isc.org/bind-keys
	//========================================================================
	dnssec-validation auto;

	listen-on-v6 { any; };
};

In this file, I would just uncomment the forwarders{} section and replace 0.0.0.0 with a DNS server I can trust to send my queries against. You can include multiple addresses here, one to each their own line and separated by a semicolon. Here's the above updated the way I would configure as such:

$ cat /etc/bind/named.conf.options 
options {
	directory "/var/cache/bind";

	forwarders {
		4.2.2.2;
		8.8.8.8;
	};

	dnssec-validation auto;

	listen-on-v6 { any; };
};

In this way, this named instance will use those IP addresses for DNS queries it cannot answer. Next, let's checkout named.conf.local:

//
// Do any local configuration here
//

// Consider adding the 1918 zones here, if they are not used in your
// organization
//include "/etc/bind/zones.rfc1918";

So, in this file, I would just scrap the comments and create my custom zone as I wanted to control it like so:

zone "markizano.net" {
  type master;
  file "/etc/bind/markizano.net.zone";
  allow-query { any; };
};

You can put whatever zone you want here. It won't be recognized by the broader Internet, but in your local network, it works great! All devices you point to this DNS server will get the addresses you control and set here! This configuration will create a zone we control in our DNS server and the target file in /etc/bind/markizano.net we will create that will contain the zone records for this domain. If you find yourself managing multiple domains, I find it easier to place these configurations in a subdirectory.

Domain Configuration

Now that we have our base server configuration in place, it's time to configure our domain! Let's setup a few essential records in /etc/bind/markizano.net.zone:

; ##### Markizano.NET
$TTL    600
@       IN      SOA     markizano.net. root.markizano.net. (
                     2024021517 ; Serial
                         604800 ; Refresh
                          86400 ; Retry
                        2419200 ; Expire
                           600) ; Negative Cache TTL

; ### Main Root DB and A records
@                           IN  A       127.0.0.1
@                           IN  AAAA    fe80::01

@                           IN  NS      ns1
@                           IN  NS      ns2

ns1							IN	A		127.0.0.1
ns2							IN	A		10.0.0.1

The initial $TTL 600 tells the default timeout for the root SOA or "Source Of Address". A Zone must have a root A record, so that is provided as 127.0.0.1. It also helps to have NS or "Name Space" records, so those are defined as "ns1" and "ns1" for "ns1.markizano.net" and "ns2.markizano.net".

Fun tip: Domain records that end with a trailing dot "." are the actual TLD or "Top-Level Domain", so you might sometimes see ns1.markizano.net..

Check Configuration

Before we restart the service, we can check the syntax of our configuration to ensure it's all correct. This avoids failures on restart. Let's use the named-checkconf command to check the configuration. This command should output nothing if there is no issues and return 0 in its exit code. If there are errors with your configuration, it should print where there is an error and you can correct that.

You can check the zones configuration file with named-checkzone like so:

# named-checkzone markizano.net /etc/bind/markizano.net.zone
zone markizano.net/IN: loaded serial 2024021517
OK

Once you get the OK from these commands, you should be good to proceed!

Restart Service

The next step after all the configuration is in place is to restart the service so the configuration is loaded into the daemon to be served up! Once the service is restarted, you can query against it to find out if your records are working!

$ sudo service named restart

Querying against the DNS server is as easy as nslookup, host, or dig. Let's use host:

$ host -d markizano.net 127.0.0.1
Trying "markizano.net"
Using domain server:
Name: 127.0.0.1
Address: 127.0.0.1#53
Aliases: 

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 27155
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;markizano.net.			IN	A

;; ANSWER SECTION:
markizano.net.		600	IN	A	127.0.0.1

Received 47 bytes from 127.0.0.1#53 in 0 ms
Trying "markizano.net"
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 57057
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;markizano.net.			IN	AAAA

;; ANSWER SECTION:
markizano.net.		600	IN	AAAA	fe80::1

Received 59 bytes from 127.0.0.1#53 in 0 ms
Trying "markizano.net"
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 41693
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;markizano.net.			IN	MX

;; AUTHORITY SECTION:
markizano.net.		600	IN	SOA	markizano.net. root.markizano.net. 2024021517 604800 86400 2419200 600

Received 72 bytes from 127.0.0.1#53 in 0 ms

and Boom! You now have a fully functional DNS server that you control! Congratulations! If you want your system to use this by default, simply update /etc/resolv.conf, make sure this entry is first in this file:

nameserver 127.0.0.1

Or whatever the IP address is of the host running your DNS service in your environment. That's how you roll your own 😎

If you like tutorials on CloudOps/DevOps stuff, feel free to subscribe to this or any of my socials! I have a lot of work to catch up on!!!
Drop a comment if you would like another post on how to secure and harden your configuration to have a DNS server face the public.