Boolean logic and Machine Learning

Brian H. Russell, Kristopher A. Innanen

Starting with a simple four-point gas and wet sand AVO cross plot example as an analogue we introduce the concept of the Boolean XOR function and show how Boolean algebra has underpinned both the development of the modern computer and the evolution of machine learning. As we show, the XOR function of Boolean algebra is nonlinearly separable but can be built from a combination of simpler linearly separable Boolean logic functions such as the AND, NAND, OR and NOR functions. We first discuss the problem in the context of Boolean algebra and then show how these concepts were instrumental in the development of electronic logic gates, which in turn were used to build the first electronic computers. The XOR logic gate is at the heart of the half-adder circuit, which gave the modern computer the ability to add (and then multiply) numbers. Simultaneously with the development of the modern computer, the mathematics behind artificial intelligence (AI) and machine learning (ML) was being developed. The first machine learning algorithm was the perceptron model, and this algorithm was successful in solving linear Boolean logic functions. However, the perceptron failed to solve the XOR problem, and this led to the first “AI winter”, in which confidence was lost in AI techniques. This was solved by adding more layers to the perceptron model, which initially was called the multi-layer perceptron, and now is more commonly called the multi-layer neural network. This approach was a direct analogue to the development of the XOR gate in electronics. In this tutorial we will first use both linear algebra and intuitive methods to derive the weights in both the perceptron and multi-layer perceptron. These weights solve the problem exactly. We will finish with a discussion of the perceptron learning rule and back-propagation technique, which solve for the weights iteratively and formed the foundation for machine learning. As will be shown, the Machine Learning solution is approximate, but will converge to the true solution as more iterations are performed.