Notice: This article is maintained for historical purposes. The Bixly Automate/Nebri OS platform that Bixly built, and this article features, is no longer available. Checkout Serverless Providers as an alternative.
We have a couple options for implementing a rules engine in Python: we can roll our own, or we use an existing one. Regarding the first option, it seems a little pointless. If you don’t give a rule engine proper respect, you might believe Python is already a nearly complete rule engine, and therefore you can easily create one from it. “See, every if statement is a rule!”
if temperature > 90:
# do something
However, having a rule doesn’t mean you have a rule engine.
A rule engine is an alternative computational model where a “rule” is king, not the execution path of the software
Creating our own rule engine looks attractive if you don’t understand the architectural shift away from traditional software. A rule based approach changes everything about execution flow in your program. It seems that many people don’t understand this. Here we see a very smart S.Lott with the following opinion on the matter:
Do not invent yet another rules language.
Either use Python or use some other existing, already debugged and working language like BPEL.
Just write your rules in Python, import them and execute them. Life is simpler, far easier to debug, and you’ve actually solved the actual log-reading problem without creating another problem.
These are three main positions here that I would like to respond to:
- Find a tool (Always a strong option. We will explore this a bit.)
- Just use Python (Sorry, wrong computational model.)
BPEL isn’t going to cut it. It’s an imperative workflow execution language that sometimes have rules added to it. You would run into the same problem as with vanilla Python: Any rule you write exists, but doesn’t necessarily matter.
A rules engine is all about providing an alternative computational model. Instead of the usual imperative model, commands in sequence with conditionals and loops, it provides a list of production rules. Each rule has a condition and an action – simplistically you can think of it as a bunch of if-then statements.
– Martin Fowler
Writing a rule in Python is not the same as writing one in a rule engine.
What about existing packages? Pyke is interesting since it creates “Expert Systems” and has an inference engine. You can find more Expert Systems here. People have been around for quite some time writing rule engines with different trappings. Business Rule Management is a fairly popular space that uses rules. Machine automation is another. Big boy tools like Drools exist to fill these needs. Just within the Python community alone we have a few options. It’s for that reason this article is not a tutorial on how to write a Python rule engine. They already exist. This article is exploring a rule empowered approach to building software. Your needs might perfectly be met with an Expert System, but that didn’t cut it for me for various reasons, so lets start there.
Expert Systems Are Not Good Enough
- Many tools are unused/not production ready
- Enterprise centric (requires large team to maintain)
- Institutionalized, focused on academia
- Learning curve is HUGE and implementation complex
- Little help with interfacing to the outside world
- You have to bring your own platform
- Little help with troubleshooting/authoring/administrating rule execution
- Algorithmic efficiency/scaling prioritized over speed of programming
I didn’t need an inference engine backed with the RETE(1) algorithm with backward and forward chain reasoning. I wanted rather to use the power of Python if-then rules to develop software faster with rules. This approach is closer to human reasoning which is attractive. I wanted to dictate, in simple modular rules, and in less lines of code, what I normally would have built with traditional software. This took me on a three year journey and a crap load of money spent. But for a good reason. You see…
Rules Are Lame Without A Platform
Think about it. Every programming language just executes rules in a way, but you are interested in having much more power. Having a place where a rule is written once and the entire environment reacts to it. That’s a platform not a library! Say, 1000 sensors being subject to one rule: if they get over a certain temperature, issue the shutoff procedure. No, take it a step further: If more than 10% of sensors stay over 55 degrees Fahrenheit for more than 15 minutes, initiate halt and alert the manager. Or, if the bounce rate gets above 60% for any page, let me know which page it is. You can even think of home automation easier this way. But rules need a platform to be fully realized!
I wanted to write pure Python rules and produce a working peice of software from them. I wanted a GUI for certain things, a debug mode, API endpoints, process histories, quarantines. I wanted one simple rule/script to wield the power over every event (via key value pair signals) that hit the system, every time. Add to that quarantines, queues, instant deployment…and more. Way more.
See it gets really complex when you think about the nature of each event hitting the system. For instance, what happens when the SECOND temperature event reaches the system? Each instance of the events need their own process ID. A cleaner example is when you use this (like we do) for tracking employees and customers. The same events can hit the system at various times representing different employees. Say you have a process to buy a computer for a new employee: IF start date > 30 ago and computer_id is NULL, computer_purchase = True. That would fire a notification off to the sysadmins. But if ever hire more than one employee these events need to remain completely separated, and be fully traceable, and use the same rules. Hence the need for a process ID system for all incoming events. Some events trigger new processes id’s if they aren’t related to an existing PID, while other events already hit the system with an associated PID.
Want more reasons rules need a platform? What about when you want to communicate between these processes ID’s or query them? For example another set of rules can watch your budget and decide if the computer account will need a deposit soon. A free standing rule by itself can not do these things.
We ended up building our own event/rules platform as an entrepreneurial effort. Given all the other IOT/Rule Engine/Business Rule Managers/Event Platforms out there, currently Nebri is the quickest way to build a rule engine centric application in Python. Also, early indications show it’s very fast at building software too.
So How Do We Build Rule Centric Software?
Pie! Here’s a demonstration of how little scripting it takes to have a fully functional app. In this example we are checking for uptime of a website:
listens_to = ['raise_if_url_down']
# pass check if url is down
r = requests.get(self.raise_if_url_down, verify=False)
if r.status_code == requests.codes.ok:
return False # all good, no action to take
return True # things are down so notify
self.url_down = send_sms(shared.managers, self.raise_if_url_down)
You just need to enter a drip into the admin, which “drips” the KVP’s into the system to be checked at the interval you prefer. See in this single script inside of your Nebri instance you have the all the trappings a full system! You have a record of checks, success or not. You have queue management if too many checks hit the system at once, or to many notification go out at once. You have an admin. You have ACL, so certain people can be notified or not. You can easily add more script to create contact escalation rules so you don’t get woken up needlessly.
To enhance this website checking system even more, we can implement rules to check certain sites at certain times, or maybe map checks to happen from different geographical locations through your other servers. It just doesn’t matter. You add a rule, and the rule is always observed.
You can implement all sorts of Python rules with VERY little code. How about turning on lights, organize your standups, react to Dropbox file changes, and WAY more.
Oh, and a closing note. Maybe you wanted your Expert System AND a great Event Based platform (maybe not, but follow me). Fear not! Go ahead and install PyKE or PyCLIPS into your Nebri environment. It is a platform after all.
1. There are two main types of rule engines algorithms: RETE and Basic. By basic I mean, basically non RETE. If you are building your own engine, you will want to figure out which path to take. RETE is complex algorithm which employs i/o nodes and trees to optimize matching efforts. It’s faster and less ram efficient. There are variations on RETE also that help bring it up to snuff. A basic algorithm takes signals and evaluates them as they come. It’s slower, and can be more ram efficient.