Audcom
Radio Silence

In 2017 so far, I've written a couple of posts that have not yet made it to the publish stage. One entitled "Fixing BSDgrep," and another with general thoughts on Python. These will eventually get published. =) Here's a little bit of what I've been up to in 2017:

Ham Radio

I recently obtained my general license in the USA, and I have plans to shoot for extra in the short- to mid- term. I have a couple of Baofengs that I use, and I've also setup a Raspberry Pi (running FreeBSD, of course) with an RTL-SDR dongle plugged into it running Direwolf to capture APRS traffic.

Fixing BSDgrep

Also the subject of one of the articles I'm writing, I've spent a good chunk of time fixing BSDgrep in FreeBSD. There were many outstanding bugs open against it in Bugzilla, but I've slowly squashed each one and got a couple of improvements through in the process mostly so that I don't have to specify a working directory with -r. I've also been doing some work in other areas of the base system to help my workflow in some areas. I intend to continue trying to improve things in FreeBSD; mostly because I get a certain satisfaction out of doing so, but also because I converted al of my stuff to FreeBSD roughly two years ago now.

Server Migration

I've also managed to migrate my www droplet with DigitalOcean to another droplet, but this time with more jails. In the new setup, I've got four jails using iocage:

  • db0 - Postgres jail, runs postgres 9.6, houses website databases
  • vpn0 - OpenVPN server for my private network 
  • www0 - Nginx jail, houses acme/letsencrypt bits
  • xmpp0 - Jabber server, runs ejabberd

This is a marked increase from the one jail (xmpp0) I previously had, but SSL certs are still mounted into individual jails as required.

Future Plans

The year is still just under half-way over, and I have much to do. Here are some future plans I have for this year:

  • Help get HDMI up on pine64/pinebook
  • Continue FreeBSD development
  • Replace old SunFire build server with more cost effective solution
  • More programming

Not the most difficult lit of things to accomplish, but still worth formalizing. I reserve the right to accomplish more or less as needed.

Recent Projects

In my previous post, I gave a brief overview of what I'd done in the past year. Looking back, though, I completely omitted recent projects that I'd written to make my life a little easier. There's only two (2) of them, so this won't be a necessarily long post.

freebsd-pkgbase-build (GitHub)

The first most recent project that I've worked on was freebsd-pkgbase-build. The gist of it is: I have multiple machines that I've taken the time to trim down the GENERIC kernel config to only include what I really need, and my machines span multiple architectures (amd64, armv6, arm64) and sometimes multiple branches (stable/11, head, stable/10) .

I wanted to try out FreeBSD's PkgBase, which is still not quite ready for prime time, but doing the buildworld's and buildkernel's for everything was kind of time consuming, not a great process, and required some maintenance overhead that I didn't necessarily want to go through. Ideally, I could:

  1. Spin off all of my pkgbase builds
  2. Create the different repos that arise from different ABIs and different branches
  3. Combine the different repos into one that I could serve fairly easy, despite these things being scattered all over the place

And thus, freebsd-pkgbase-build was born. It is a make(1) based solution that uses a Makefile.local or /etc/make.conf for configuration. My Makefile.local looks like so:

SRCTOP+=/usr/src.head:amd64.amd64
SRCTOP+=/usr/src.11:arm.armv6

MAKE_ARGS_arm.armv6+=   MALLOC_PRODUCTION=yes

Rather intense, yes? No, not really. My arm64 stuff is excluded right now, but it's a fairly simple addition. After inclusion of this, it's a simple `make packages` to produce the repo at /usr/local/pkgbase/repo. Easy peasy, yeah? Yeah.

py-rtprio (GitHub, PyPI)

py-rtprio is my other project. It is also nothing fancy, just a Python-based interface to rtprio(2) for setting realtime priority of the executing thread/process. Originating in Python 3, shortly ported to be Python 2 compatible just to say I tried to be friendly. Admittedly, I like the Python 3-only version better, but it didn't end up too terrible. It uses ctypes to load libc and expose rtprio(2) in a Python-ish way. When I decide to drop Python 2 support, I'll likely go back to the cleaner version that uses Enum, which I avoided for Python 2 support so that I didn't need to bring in a dependency just for that one small (yet nice) feature.

My motivation for this was that I have many dreams of doing things with FreeBSD that are more suited to an RTOS, but I think with setting real-time priorities I might be able to accomplish some of the things I'd like to do. My reasoning (likely naive) is that setting a real-time priority on a system that I already tightly control will limit the number of things that can pre-empt me to just the kernel. My hope is that will net me the responsive-ness I may require.

Desert Wasteland

Hi,

Wow, it's been almost a full year since I've written anything. I guess it's time for the yearly update, then?

FreeBSD Shenanigans

In the past year, I've been involved in more FreeBSD shenanigans. Find enclosed a list of highlights.

Servers

In my last post, I'd migrated my DigitalOcean droplets to FreeBSD.  Since then, I've pretty much consolidated all of them into my one webserver that runs all of my websites and a jail for Jabber. Want to contact me? JID=kevans91@audeuro.com is just one more way to do so. =)

I've also stood up a Sun Fire X4150 (with dual Xeon E5440, 24GB RAM) at home with the following roles:

  • Poudriere Build System
  • IPCam Manager
  • OpenLDAP Server
  • Personal wiki (Running DokuWiki)
  • Personal Git server (Running my own git-shell wrapper, gitwrap-shell)
  • Jenkins Builder (For keeping tabs on ZDoom build status on FreeBSD as the maintainer)

It stays fairly well loaded, and I'm considering standing up a second server (ideal Sun Fire X4150) to operate as a backup server. Most of my stuff uses ZFS as it is, but getting some redundancy going would be quite nice.

Ports

I've also tried to stay fairly active with the ports tree this year. The things I've done here include:

  • Took maintainership of net/freerdp
  • Implemented arbitrary baud rate support for *BSD in comms/py-serial - upstream first, then back to the ports tree
  • Provided distfile mirror for games/bsp and games/heretic
  • Got fmodl aliasing issue on arm, mips, powerpc resolved
  • Introduced 16-ish ports total, most of them for Arduino 1.6 (devel/arduino16) support, which brings along SAM support (Arduino Due)
  • Introduced OctoPrint (www/py-octoprint, loosely based on the 1.3.0 release) for interfacing with my work-in-progress 3D printer (which I will likely talk about in a future post, with pictures!). I've contributed upstream some bits to update dependencies to bring OctoPrint to FreeBSD -- we'll see where my upstream PR ultimately goes. I remain hopeful.

You can review the list of ports that I currently maintain (under the assumed alias of bsdports@kyle-evans.net -- sssh! Don't tell anyone!) if desired. Most of these were introduced for bringing in the new Arduino IDE.

Hardware

Worth noting is that I've spent some amount of time playing around with different hardware. I've deployed two FreeBSD Raspberry Pis, one to go along with my work-in-progress 3D Printer and another that does...well, not much of anything but sit on my desk at work.

I also bought a Carambola 2 Dev Kit to hopefully replace my router or my AP/extender at home. I put FreeBSD on it using Adrian Chadd's freebsd-wifi-build, but ran into problems when attempting to isolate the WAN port on the chip. Attempts to setup VLANs ended up in interrupt storms and failing to TX on the port with a different VLAN. One day, I might find time to debug this.

Since that didn't work out so great, I decided to put FreeBSD (again, using freebsd-wifi-build) on my TP-Link TL-WDR4300. I ended up in pretty much the same situation as before -- the case physically separates one of the ports as the WAN port, but the hardware doesn't really make this distinction. There's 4 LAN  + 1 WAN port, but underneath it's really just a five port switch with some clever tricks to isolate the WAN port. I've consulted Adrian on this setup because I'm having difficulties again isolating the WAN port, I will update this post when I hear back from him. My current problem seems to be that I can't TX on any port again no matter the VLAN configuration I try. I assume I'm handling the cpu ports on the switch wrong, but we shall see.

My other experiment was with my IP Cameras. I have two of them, both Microseven models. I'm not very happy with the WiFi stability on them, so I intend to flash them with FreeBSD. I disassembled one of them to see if I can't get a serial console on it, but failed at this, too. My first road-block was that what I've identified as the serial pins are 1.27mm pitch, but all of my headers and cables are 2.54mm. I ordered 1.27mm headers from Alibaba, all OK. However, I don't actually have cables I can work with 1.27mm pitch with. =( It's a work in progress.

The other hurdle on that one is that the WiFi chip seems to be an SDIO chip that we don't currently have support for in FreeBSD. I have no further details on it until I can actually boot with a serial console, unfortunately, so we'll find out more once I can work with that.

Personal - Graduation / Work

I graduated from Kansas State University in May with a B.S. in Computer Science. I did not get a minor at this time, because I was just ready to be done. I took a full time position at Kansas State University after that, and continue to do, for the most part, what I was doing before I graduated. Student loans are coming due right about now, though, so that's not so great.

Conclusion

I think I've said all that I need to for this year. I can say that I really did do things this year, although they may not have been the things I wanted to accomplish. I hope to do a little more writing in the upcoming year than I did this year, and use more pictures to complement my writing. At the very least, I need to talk about my 3D Printer as I bring it to completion as well as documenting the journeys in getting any of my more exotic hardware running FreeBSD.

Servers, Ubuntu -> FreeBSD

Hello!

There's actually a couple of intermediate steps that have happened in the past year and a half or so. Arch was falling out of favor with DigitalOcean, so I moved my servers over to Ubuntu. As one might expect, that migration was relatively painless. There were version differences between the two, but for the most part my configuration and setup could be copied almost as-is and everything worked pretty well.

For the move to Ubuntu, I did alter my servers a little bit. My webserver picked up Varnish in front of it, and I believe this was also the point where I made the full switch over from Apache to NGINX, instead of maintaining a hybrid setup with my Drupal sites being on Apache and my non-Drupal sites being on NGINX. I put together a PHP-FPM setup that worked well to shove out my previous Apache/suPHP setup and separated out my NGINX configuration enough that adding new sites is a breeze. Around this time, I also dropped GitLab and developed my own "gitwrap" utility w/ Python 3. My util and git servers were merged into one, and I dropped my BYOND and area51 servers as they saw very little activity. I also dropped OpenLDAP for the time being, as I was no longer running Redmine.

That brings us up to November 2015. As of September, I had switched my Desktop over to FreeBSD. I eventually switched over to Xfce, and I've yet to look back. In November, I decided to take the plunge and convert my servers from Ubuntu to FreeBSD. The initial step was to assess version differences between Ubuntu and FreeBSD. Much to my fortune, there was not much of a discrepancy in Varnish or NGINX versions-- nothing to significantly change my configuration, the following steps were then taken to migrate properly:

At first, I put up two servers that I would then be moving to: www2 and util2. Only being two months into my FreeBSD experience, I hadn't actually setup any of the stuff I would need on my servers before in a FreeBSD-way. So, I installed www/nginx, lang/php56, databases/postgresql94-server, and www/varnish4 (FreshPorts is awesome, by the way).

First observation: php-fpm actually comes with lang/php56...OK, that's interesting (and convenient)! I was already aware that most configuration of packages not in the base system occurs in /usr/local/(etc), so I was prepared for that. I wasn't quite prepared for php/fpm conf files to just be dropped into /usr/local/etc, but that's not exactly a killer. The stock php.ini-production was suitable for my needs, so I symlinked that over to php.ini and moved on to FPM.

For FPM, life's not too difficult. Adding an "include=etc/fpm.d/pool.d/*.conf" is enough for me to go ahead and copy over my pool definitions. For each pool, very little ultimately needed changed. Listen socket as well as user/group and listen owner/group, then log locations to reflect that I'd now like to put logs in log/nginx (/usr/local/log/nginx).

After that, FPM and PHP were happy enough that I went ahead and setup NGINX. This setup was also pretty easy. I commented out the server {} block in nginx.conf and added an include directive to make sure I was pulling in etc/nginx/sites-enabled/* -- which is where I prefer to store my server {} blocks. I pulled this one from Apache habits, and have a corresponding sites-available/ directory that holds all of my site configuration, and then sites-enabled/* is just a bunch of symlinks. Other than that, I was actually able to drop in my drupal.conf, php.conf, set-pool.conf, and ssl.conf to etc/nginx without changing anything except the path to my php-fpm sockets in my php.conf. SWEET! Now NGINX is happily serving my domains, although I hadn't quite switched over DNS at this point or stood up Postgres.

Varnish was also relatively easy-- still using Varnish 4, so I could drop my 4.vcl from my Ubuntu configuration into etc/varnish and setup varnishd_config in /etc/rc.conf appropriately, and all was good.

Postgres was a slightly different story. I was moving from 9.4 to 9.4, so things got a little bit sketchy. I've got 9 or 10 different databases in the same cluster, so I didn't necessarily want to backup/restore to a new cluster as I probably should have done (note: If it's feasible, this should be the recommended method- especially if you're moving between minor versions). Instead, I rsync'd my data/ directory over on a live/running instance of Postgres, and adjusted accordingly.

I did not maintain symlinks to any of the *.conf files as I was in Ubuntu, but that didn't seem to be an issue. I created new local users for my corresponding database users, and then ran into an issue with postgres/pgsql differences. In Ubuntu-land, the user is named 'postgres' -- this seems to make more sense to me, because initializing the cluster seems to automagically created a databasee named 'postgres'. databases/postgresql94-server actually creates a 'pgsql' user, but cluster creation still automagically creates the 'postgres' database- so a `psql` as pgsql won't actually get you into a working database or command-line at first, which seems kind of odd. If I were first dealing with postgres on FreeBSD without any previous experience, I would probably be mildly confused by this. I tested this out on my laptop, just to make sure I wasn't crazy.

Much to my happiness, though, this pretty much worked. All of my data happily carried over, I just needed to get in and rename the 'postgres' user so that my local authentication as 'pgsql' would still be happy. My webserver was up and running at this point, other than needing to install a couple of other extensions (all suitably named either php56-* or pecl-* -- this made things really easy to work with).

My git server is not much to speak of, however. As it was a custom 'gitwrap' wrapper, all I really had to do was install lang/python35, rsync my gitwrap directories over, and then everything was pretty much happy.

I suppose the point (or TL;DR, if that's your flavor) of this post is that it doesn't have to be painful to make the switch from Linux to *BSD. Granted, my setup isn't all that intense/extensive- I have Drupal (backed by Postgres) running w/ PHP, PHP-FPM on NGINX behind Varnish. I have a couple of other odds and ends running around on here, but nothing to speak seriously of. My webserver doesn't have an entirely uncommon setup, and I was still able to easily find comparable ports to suit my needs. Installing PHP extensions was also entirely straightforward, and these things are pretty well-named. The problems you run into in your own migrations will be different than the problems I experienced, but given what I've already seen and know- I don't suspect they'll be anything major or show-stopping. My migration for both servers was completed in under 3.5 hours.

<3 /var/backups

Long story short, I got tired of Windows 10 a couple of weeks ago. The performance on my five year old dual core laptop was less than satisfying, and I wanted to get more out of my laptop. Enter FreeBSD.

I started out with 10.2-RELEASE and KDE4. It had some annoying issues, particularly in switching to a virtual console locking up the system. It didn't take long for me to decide there's not much reason to stay on -RELEASE, so I moved to -CURRENT. This background comes in now because it may be relevant in future posts about FreeBSD exploits.

Anywho! In my weekly update to the latest -CURRENT && rebuilding world, I started running mergemaster. I was in a little bit of a hurry and managed to nuke my /etc/group and overwrite /etc/passwd. Cue a good six minutes figuring out an easy way to restore all of the passwords and groups I lost, until I came across a post mentioning that there should be backups of my previous revision in /var/backups. Restored all, happiness returns, thank you internet.

Servers

Shockingly enough, I have a very large interest in technology. From electronics to programming to system administration, I'd like to be diverse enough to have confidence enough in all three of these areas. The latest area I've been developing in is system administration.

At my current job, I do not get much exposure to doing system administration. I do, however, get to be exposed to a lot of the inner-workings that drives our network. This gives me a general idea of network architecture and how enterprise-level systems should really work. We have a lot of servers doing a lot of different things, and most of them are pretty much independent of each other and secure.

This gave me ideas for my personal setup, running this website and a couple of other services, causing me to land on the following servers:

  • www - The sole purpose of this server is to run any websites that I maintain. This includes the one you are currently viewing as well as http://www.openproto.org [work in progress]
  • util - This is my utility server. It runs my LDAP server and it will eventually run my mail server. LDAP is only accessible over the private network, and mail will function the same.
  • area51 - This is my development server. It is for projects that I'm working on that are not quite ready for production. Once a project is ready for production, all trace of it is removed from area51 and it is migrated either to one of the other servers that suits its purpose or a new server will be spawned for it.
  • byond - This is my BYOND server. It is special because it is running a 32-bit OS as opposed to 64-bit. It will not only power any production BYOND applications I have, but it is currently setup to pull all of my BYOND projects that get served and builds them for me on the server every 30 minutes. Moving them to be hosted in production is then simply a matter of copying the necessary files over to the hosted directory and rebooting the server.
  • git - This is my git server. Its sole purpose is to run GitLab. It is special because is my only server that is NOT running Arch. It is instead running Debian, which is one of the recommended distros for GitLab.

All of my servers run 64-bit Arch, unless otherwise noted. In the future, I plan to introduce a Windows server (just to try it out), and servers for at least one or two more products that I currently have in miind.

Introduction and Statement of Purpose

Hi there, this is my personal website. It shall serve as an index for my published(-esque) projects, as well as a place for me to rant about some of the aspects of development that I find to be 'cool' or otherwise interesting. One of the first of such posts will probably be about version control and why it's so awesome.

As a bit of background about myself (some of which is on the /whoami page, so excuse the redundancy), I am a student in Electrical and Computer Engineering at Kansas State University. I have been programming since 2004 (~9 years, as of the time of writing) in various languages. These languages include, but are not limited to:

  • Visual Basic 6
  • BYOND/DM
  • C++
  • C
  • Java
  • C#
  • PHP
  • Javascript

This list is in the order in which I picked them up, rather than any order of preference. My favorite language of the above listed is C++, and my favorite platform is Linux. My favorite flavor is Arch, which I use everywhere- including my netbook and on the server running this website. I've been working with Linux in various capacities since 2002 or 2003, and I've been doing administration of Debian-based servers since 2006 or 2007, with a small hiatus from everything for a year or two starting in 2008 when I moved to Kansas and ending in early to mid 2009.

As far as C++ goes, i've been working with it since late 2004 or early 2005, and I've still got a lot to learn. Most of my experience in it has been with networking (mostly playing around with Berkeley sockets, because Winsock makes me cringe), but I've also dabbled in OpenGL and playing around with Qt/SDL/Fun stuff like that.

Well, that's all for now. Until next time.