| title | keywords | ||||
|---|---|---|---|---|---|
Why CRUD Applications are hard? |
|
CRUD, or Create-Read-Update-Delete, are web applications aimed on display and processing of information. They allow users to create, modify and browse some information. These are your ordinary forums, blogs, online shops, news websites etc.
Such an application sounds simple to implement. Yet in practice, I found implementing them to be time consuming. In this post, I am trying to explore why. I have not came to any definite conclusion or proposal yet, so these are just my thoughts on the matter. Possible solutions are considered in the context of Scala, so the libraries referred here are Scala libraries.
Since CRUD applications are so abundant, it is natural that there exist a lot of frameworks and libraries for their implementation. Let us briefly overview what is available.
CMS, or Content Management Systems, are probably the easiest way to implement CRUD applications. They provide a ready website aimed at processing of a specific kind of information. For example, Wordpress aims at blogging. It knows how to store blog posts in a database out of the box and provides you with a blog-aware backend. Shopping CMSs like Prestashop know how to work with goods, customers and orders.
The strong side of CMSs is a lot of capabilities out of the box. After installing a CMS, you get a working website that you can configure and work on without any knowledge of programming.
The weakness of CMSs is lack of flexibility. While you are working within the assumptions of a CMS, everything goes smooth. But whenever you need additional features, you may need to resort to plugins. Sooner or later you will encounter a need that is not addressed by any existing plugin and you'll need to write your own. To do so you'll need to learn the CMS architecture and how its plugins work. This may imply a steep learning curve. Worse, after learning a CMS, you may find that its architecture is not so convenient to work with, or that your feature can not be implemented at all.
These require you to describe your website in code. But they also dictate you the rules according to which you should program. If you obey these rules, your framework automates many things for you. Some examples from the Java/Scala world include Java Servlets, Spring, Scala Play, Scalatra.
Frameworks abstract away technical details so that you can focus on what really matters to your application. For example, instead of describing how to receive a web request and how to parse its headers you should be thinking about the business logic of your application - what happens when a user posts a message to the forum.
The problem with the modern frameworks is that they don't abstract the right thing. They mostly focus on hiding the complexity of the inner workings of the server. Some of them also provide abstractions for templating, validation and other similar things. But is that enough? If not, why and what do they fail to abstract?
Consider building a forum. What will this task involve?
First, we need to model the domain. We have the following data types:
- User
- Thread
- Message (in a thread)
Next, we need a way to store and query these in a database. We need to create a database schema and a data access object for each of our entities.
We also need to specify the business logic, the API of our application. What can the users do and what happens on every action? We can support something as follows:
- Registration and logging in.
- Browsing all the threads.
- Browsing a particular thread with comments.
- Creating new threads.
- Creating new comments.
- Moderators can also update and delete comments and users.
We can implement this API by defining some service objects.
Finally, we will need HTML views and HTTP request handlers. The handlers will perform API calls and render their output using the views.
So, the model objects, the DAO, the API, the views and the handlers - these are 5 steps to do to add some new data-related functionality to the site!
Yet, a lot of the above information can be derived from the model objects and the API logic.
Programming is an art of abstraction. In the above example, what is technical details and what is something we'd like to focus on?
Let us look at some web apps out there. Online stores. Forums. Blogs. News websites. What do they have in common? How are they different from desktop or mobile applications? What are they all about?
- One server with some information on it. Blog posts, forum posts, news - all this is information the server stores and modifies.
- Many users accessing and modifying this information.
- Access via HTTP protocol. These users access and modify the information via HTTP requests and responses.
So a web application is a program that persists information and defines how users can modify it via HTTP protocol.
If a web app is all about how to store and process information, probably this is what a programmer should focus on when writing it.
Under this approach, we should aim for abstracting everything that does not describe the way the data is accessed and processed. Ideally, it would be nice to have a DSL that would allow to specify the data processing declaratively.
What is it possible to abstract away?
First, DAO and persistence. They have nothing to do with the business logic of the application, so we do not want to focus on them. Given a model object, it is possible to derive its SQL schema and CRUD methods to access and modify it.
Next, HTML views and HTTP handlers. These define an interface to the API methods. Based on the API methods themselves, it may be possible to generate the handlers and the views.
Scala macros and techniques for generic programming can be an answer here.
Macro annotations can analyze domain objects on compile time, similarly to how Freestyle utilizes them. They can then generate all the boilerplate DAOs with CRUD methods.
If an object has relationships to other classes, these relationships can be detected (e.g. non-primitive field in a model class means a relationship) and read methods can be generated for these too.
If the API methods are mostly about CRUD operations, here is how we can generate HTTP handlers and views from them:
- Create and Update methods generate two pages. One page contains a form a user can fill to specify the modifications they want to make. The other is the handler page that takes this data and calls an API method with it. The forms can be derived via the API method parameters.
- Read methods generate one page. Parameters to these API methods can become REST or GET request parameters. Reasonable default views can be derived from the types these API methods return.
Analysis of the API methods can also be done via macros. All the API methods have some input and output types. We can annotate the API methods with @get and @post annotations: if it is a GET method, the input parameters will come from the GET HTTP request, if it is a POST method, then a separate page with a form should be generated for it.
The GET handlers will just render whatever the API method returned, the POST handlers will use redirects to display the result of the rendering.
API methods should be converted to HTML views in order to get rendered. Also HTML forms should be generated for POST requests to provide input values for the API methods.
For prototyping or simple websites, making views by hand can be tedious. It would be nice to have an option to generate them automatically based on simple rules.
For this purpose, Shapeless can be used. It can easily generate views for complex case classes based on the views of the primitives it is composed of.
It would be nice to have a framework that allows to easily generate CRUD applications. It should be flexible, free you from the boilerplate, but it should also allow the programmer to have full control over the application. It should be possible to turn off or override any part of the framework at will.
Modern solutions are either too large, or too inflexible, or both. Therefore there is a need for something more compact that allows both for easy prototyping and for a production-ready application.