Algorithms of the Intelligent Web

One Minute Bottom Line

If you want to learn about technologies that help improve the intelligent behavior of your applications, this book provides you with a good overview of what is currently avaible in technologies and products. Although the market value of intelligent applications is huge, this book is not about marketing. It provides high-level mathematical overviews of technologies, that are not too difficult to follow though.

Review

This book is relevant for developers and project managers that want to add value to existing and new projects by using intelligent algorithms. It is not a marketing book, focus is clearly on the different technologies and how to choose and implement one.

This book describes a number of techniques that can be used in intelligent applications. Do not expect to find a silver bullet inside. To the contrary, this book clearly shows that making applications more intelligent and making their behavior dependent on the input is not an easy job. However, a large number of technologies and algorithms already exist and developers can benefit from them. This book gives an overview of some of the most important technologies and algorithms that are available for implementation today.

After reading this book, developers and product managers should have some ideas about candidate technologies that their application can benefit from. Although the authors often go deep into mathematics, further reading is often recommended in order to investigate what is currently available in the market.

The first chapter gives an overview of a number of intelligent web applications. It also lists a number of applications that can benefit from intelligence. This is important to a larger audience since most of the applications have at least something to do with social networking and user collaboration.

Each of the next chapters discusses a particular technology that can be applied when making applications intelligent: searching, recommendations, clustering, classification and combining classifiers. Each chapter comes with a number of code samples and some to do's.

Chapter 2 discusses searching. It starts with the rather traditional approaches and follows with the more advanced and intelligent technologies that will improve the end user satisfaction. The basic concepts in this chapter are demonstrated with the Lucene search project and the PageRank concept of Google. It is shown how taking user clicks into account can improve search results.

Chapter 3 discusses recommendations. This is a concept most Internet users are familiar with, and it is one of the highest valued concepts on the Internet, e.g. Amazon book recommendations. The chapter starts with explaining the concepts of distance and similarity, the core concepts in typical recommendation engines. Recommendations are often based on similarity between users and content. The chapter gives examples and concludes with some remarks about performance and operations.

Chapter 4 explains the importance and benefits of clustering. It starts easy with simple SQL-based clustering examples. Next, the more advanced techniques of clustering are discussed, and a number of algorithms are explained: single link, average link, MST single link, k-means, ROCK and DBSCAN. The authors clearly describe the advantages and pitfalls of the different algorithms. This should help readers choose the appropriate algorithm for particular applications. The chapter concludes with some practical aspects encountered when dealing with large datasets.

Chapter 5 discusses classification and explains techniques for putting data and users in the most appropriate category. This chapter contains a justification for its presence in the book, which is very interesting, but only towards the end when it gets to the point that is applicable to the Internet. A link with the Semantic Web is touched briefly. Different algorithms for classification are explained. Important examples discussed in this chapter include spam filtering and fraud detection. The chapter concludes with practical remarks when dealing with large datasets.

Chapter 6 is a follow-up on chapter 5 and discusses the combination of classifiers. A case study used throughout this chapter is evaluating the credit worthiness of a user for a mortgage application. A combination of classifiers can give a much better result, but the authors make clear that this is very dependent on the nature of the particular problem and the dataset. The chapter briefly touches more advanced methodologies, including bootstrap aggregation and boosting.

At first sight, this book is rather theoretical. Indeed, a number of models are built using core mathematical and statistical principles. Some mathematical background will definitely help in understanding the algorithms. However, the authors did their very best in combining the theoretical background of intelligent algorithms with practical use-cases. After reading the book, readers will have a better understanding of different kinds of algorithms, the pro's and cons of each technique, and the applicability of the algorithms to their particular application. The examples that are given in the different chapters are often very applicable to situations every developer is facing in their career, without realizing that a wide range of algorithms exist to help solve problems.

Another prove that the book is not just a theoretical guide, is the fact that the authors included a number of issues that will be encountered when dealing with large datasets. Most intelligent algorithms rely on large datasets, and it would make no sense to come up with an algorithm that requires more computer power than can be justified by the benefits.

Although chapter 1 and chapter 7 are more general, there could have been more emphasis on the importance of intelligent algorithms from a business point of view. The quality of searching, recommendations, classification, etc. is often extremely important to the success of a project. The relevance of intelligent algorithms in todays and tomorrows Internet projects can hardly be overestimated.

In the preface, the main author states that “the goal of this book is to equip you with a number of techniques that will help you build intelligent behavior in your application, while assuming as little as possible with regard to mathematics”. In my opinion, they did a great job in achieving this goal.