Saturday, November 27, 2021

I'm joining MariaDB!

Some weeks ago I decided to accept a position as a Developer Advocate at MariaDB Corporation. As one of the public faces in the Vaadin community, I thought it would be a good idea to tell more about this career move.

My journey with Vaadin started more than ten years ago when I was looking for a way to implement CRUD web views on top of JPA/Hibernate for a web application in a reusable way. I tell more about this in my book Practical Vaadin (Apress, 2021) if you are interested. In short, Vaadin was the web framework I was looking for. It allowed me to implement the presentation layer of a web application using my favorite programming language—Java. My enthusiasm for Vaadin opened doors at Vaadin, the company, where I moved to a Developer Advocate position. I interacted with developers from everywhere in the World on topics related to Java and Vaadin. I published three books, many technical articles, a multitude of videos, and spoke at international conferences and Java User Group events.

In the world of Developer Relations (or Developer Advocacy), you interact with many developers including fellow developer advocates. One of these interactions was with Rob Hedgepeth (Director of Developer Relations at MariaDB) on a collaboration between Vaadin and MariaDB. It was a great match since (almost) every Vaadin application needs a database and (almost) every MariaDB database needs a graphical user interface. We thought it'd be useful to show Vaadin developers some of the cool features of MariaDB, and MariaDB users how to build web UIs in Java using Vaadin. I even published some videos showcasing MariaDB ColumnStore engine well before I thought about joining MariaDB.

As I investigated MariaDB, I realized I had no idea about all the cool features it has. Things such as encryption at rest, multi-master clustering, Oracle PL/SQL compatibility, columnar storage, zero interruption failover, hybrid transactional/analytical processing, distributed SQL, and others, made me interested in learning more about MariaDB's technologies. As I continued to use MariaDB I came across a job ad for a Developer Advocate position at MariaDB Corporation. The description of the job sounded great, so I asked Rob about it. The rest is history.

I'm leaving the best company I have ever worked for. A company with the most talented people I have ever met. With the nicest people that I have met. A company that builds the best Java web framework in the market, make no mistake about it! During the latest years, I got to share the cool things that Vaadin builds and I cannot count the number of "wow" reactions that I have seen.

But I'm excited about the future. MariaDB Corporation has its headquarters in Finland, where I live. As an expat, moving from one Finnish company to another feels great. Also, the possibilities for a developer advocate are endless! You can build anything on top of MariaDB using any technology. I probably won't move away from Java but I'll be able to reach out and help more developers than before, including developers from other platforms. The journey is about to start and there are many things that I have to learn. I'll give my best to share my learnings as I go through them.

What will happen to my Vaadin add-ons? I'm the maintainer of Crud UI and Report UI, two popular add-ons for Vaadin. I'll try to maintain the add-ons on a best-effort basis. I probably won't be able to add a lot of new functionality to them but I'll try to keep them compatible with the latest Vaadin LTS version. I can definitively use some help with this! If you are interested in taking over these add-ons or contributing, drop me a line!

Thanks to everyone at Vaadin—these 6 years working with you were the best. Now it's time to say goodbye to the reindeer and hello to the sea lion!

Share:

Tuesday, September 28, 2021

Testing MariaDB ColumnStore performance

MariaDB's ColumnStore is an engine that stores data in a columnar fashion. Although it shines in distributed architectures where you have a massive amount of data, you can see non-trivial performance improvements in single-instance architectures. This article shows how to:

  • install ColumnStore using Docker in your development machine,
  • create a simple Java web application to generate realistic test data,
  • run test SQL queries to compare execution times of the ColumnStore vs InnoDB engines.

To follow this article, you need Java, a Java IDE, and Docker installed on your computer.

What is ColumnStore?

ColumnStore is a pluggable engine for MariaDB and MySQL. You can install it on top of an existing database instance. It is possible to have tables in the same database using multiple storage engines. For example, you can create a table called book for OLTP using the general-purpose InnoDB engine, and a table called book_analytics for OLTP using the column-oriented ColumnStore engine. Both tables can live in the same database schema and you can run queries that mix the two. Check these online resources to learn more about OLTP/OLAP and the different storage engines available in MariaDB:

Installing MariaDB and ColumnStore using Docker

Make sure you have Docker installed and running on your machine and use the following commands to download an image and create a container with CentOS, MariaDB and ColumnStore preconfigured:

docker pull mariadb/columnstore

docker run -d -p 3306:3306 --name mariadb_columnstore mariadb/columnstore

Configuring the database

If you want to configure the instance you can connect to the container using:

docker exec -it mariadb_columnstore bash

For example, you can edit the /etc/my.cnf file to increase the buffer pool size (this is an optional step):

[mysqld]
innodb_buffer_pool_size=1G

Remember to restart the container when you make this kind of change:

docker restart mariadb_columnstore

Connecting to the MariaDB instance

Use any SQL client that supports MariaDB to connect to the instance. For example, you can use IntelliJ IDEA's database view, or the command-line tool, mariadb. If you have MariaDB in your host machine, you already have the tool:

mariadb --protocol tcp -u user -p

Since a Docker container is not really a virtual machine, you connect to the database like if it was running natively on your matching, that is, the host is localhost.

Setting up the test user and tables

Create a new database user:

docker exec mariadb_columnstore mariadb -e "GRANT ALL PRIVILEGES ON *.* TO 'user'@'%' IDENTIFIED BY 'password';"

Create a new database schema and two tables, one using the InnoDB engine and another using ColumnStore:

CREATE DATABASE book_demo;

USE DATABASE book_demo;

CREATE TABLE book
(
    id           int(11) NOT NULL AUTO_INCREMENT,
    title        varchar(255) DEFAULT NULL,
    author       varchar(255) DEFAULT NULL,
    publish_date date         DEFAULT NULL,
    pages        int(8)       DEFAULT NULL, 
    image_data   longtext     DEFAULT NULL,
    PRIMARY KEY (id)
) engine = InnoDB;

CREATE TABLE book_analytics
(
    id           int(11) NOT NULL,
    title        varchar(255) DEFAULT NULL,
    author       varchar(255) DEFAULT NULL,
    publish_date date         DEFAULT NULL,
    pages        int(8)       DEFAULT NULL
) engine = ColumnStore;

Generating test data using Java

Create a new project using the Spring Initializr and add the Spring Data JPA, MariaDB Driver, Lombok, and Vaadin dependencies. Also, add the following dependency to the pom.xml file:

<dependency>
    <groupId>com.vaadin</groupId>
    <artifactId>exampledata</artifactId>
    <version>4.0.0</version>
</dependency>

Configure the database connection in the application.properties file:

spring.datasource.url=jdbc:mariadb://localhost:3306/book_demo
spring.datasource.username=user
spring.datasource.password=password

Create a new JPA Entity class to encapsulate the test data and persist it in a table row:

package com.example;

import lombok.Data;
import lombok.EqualsAndHashCode;

import javax.persistence.*;
import java.time.LocalDate;

@Data
@EqualsAndHashCode(onlyExplicitlyIncluded = true)
@Entity
@Table(name = "book")
public class Book {

    @EqualsAndHashCode.Include
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Integer id;

    private String title;

    private String author;

    private LocalDate publishDate;

    private Integer pages;

    @Lob
    private String imageData;

}

Create a new repository:

package com.example;

import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.stereotype.Repository;

@Repository
public interface BookRepository extends JpaRepository<Book, Integer> {
}

Create a service class with logic to generate realistic random test data in batches:

package com.example;

import com.vaadin.exampledata.ChanceIntegerType;
import com.vaadin.exampledata.DataType;
import com.vaadin.exampledata.ExampleDataGenerator;
import lombok.RequiredArgsConstructor;
import lombok.extern.log4j.Log4j2;
import org.springframework.stereotype.Service;

import java.time.LocalDateTime;
import java.util.List;
import java.util.Random;

@Service
@RequiredArgsConstructor
@Log4j2
public class GeneratorService {

    private final BookRepository repository;

    public void generate(int batchSize, int batches) {
        var generator = new ExampleDataGenerator<>(Book.class, LocalDateTime.now());
        generator.setData(Book::setTitle, DataType.BOOK_TITLE);
        generator.setData(Book::setAuthor, DataType.FULL_NAME);
        generator.setData(Book::setPublishDate, DataType.DATE_LAST_10_YEARS);
        generator.setData(Book::setPages, new ChanceIntegerType("integer", "{min: 20, max: 1000}"));
        generator.setData(Book::setImageData, DataType.BOOK_IMAGE_URL);

        for (int batchNumber = 1; batchNumber <= batches; batchNumber++) {
            List<Book> books = generator.create(batchSize, new Random().nextInt());
            repository.saveAllAndFlush(books);
            log.info("Batch " + batchNumber + " completed.");
        }
    }

}

Create a web UI:

package com.example;

import com.vaadin.flow.component.button.Button;
import com.vaadin.flow.component.html.H1;
import com.vaadin.flow.component.notification.Notification;
import com.vaadin.flow.component.orderedlayout.VerticalLayout;
import com.vaadin.flow.component.textfield.IntegerField;
import com.vaadin.flow.router.PageTitle;
import com.vaadin.flow.router.Route;
import lombok.extern.log4j.Log4j2;

@Route("")
@PageTitle("Data generator")
@Log4j2
public class GeneratorView extends VerticalLayout {

    public GeneratorView(GeneratorService service) {
        var batchSize = new IntegerField("Batch size");
        var batches = new IntegerField("Batches");

        Button start = new Button("Start");
        start.addClickListener(event -> {
            service.generate(batchSize.getValue(), batches.getValue());
            Notification.show("Data generated.");
        });

        add(new H1("Data generator"), batchSize, batches, start);
    }

}

Running the generator web application

The project includes an Application class with a standard Java entry point method. Run this application as you would with any other Java application using your IDE or the command line with Maven (make sure to use the name of the JAR file that your project generated):

mvn package

java -jar target/test-data-generator-0.0.1-SNAPSHOT.jar

The first build and run of the application take time, but further builds and runs will be faster. You can invoke the application in the browser at http://localhost:8080:

I generated 100 batches of 10000 rows each for a million rows in the database. This takes some time. Check the log if you want to see the progress.

Challenge: Try using the ProgressBar class, the @Push annotation, and the UI.access(Command) to show the progress in the browser.

Running a simple ETL process for analytics

To populate the book_analytics table, we can execute a simple SQL INSERT statement that serves as an Extract, Transform, Load (ETL) process. In concrete, the image_data column stores the image potentially used in a web application that shows the cover of a book. We don't need this data for OLAP, which is the reason we didn't include the image_data column in the book_analytics table. Populate the data with the following SQL statement:

INSERT INTO book_analytics(id, author, pages, publish_date, title)
    SELECT id, author, pages, publish_date, title FROM book;

The process should take only a few seconds for a million rows.

Comparing the performance of analytical queries

Below are some examples of queries that you can use to compare the performance of the ColumnStore engine vs InnoDB. I used an empiric approach and run each query several times (5 to 10 times) taking the fastest run for each. Here are the results:

SELECT COUNT(id) FROM book; -- 251 ms
SELECT COUNT(id) FROM book_analytics; -- 111 ms

SELECT publish_date, SUM(pages) FROM book GROUP BY publish_date; -- 767 ms
SELECT publish_date, SUM(pages) FROM book_analytics GROUP BY publish_date; -- 206 ms

SELECT author, COUNT(id) FROM book GROUP BY author HAVING COUNT(id) > 5; -- 20 s 218 ms
SELECT author, COUNT(id) FROM book_analytics GROUP BY author HAVING COUNT(id) > 5; -- 1 s 93 ms

The biggest difference is in the last query. 20 seconds vs 2 seconds approximately. All the queries show a non-trivial advantage in favor of ColumnStore.

Take into account that for other kinds of database operations, ColumnStore could be slower than InnoDB. For example, queries that read several columns without aggregate functions involved. Always make informed decisions and experiment with queries before deciding on InnoDB, ColumnStore, or other engines for your database tables.

Share:

Wednesday, September 8, 2021

Using Vaadin web components in HTML documents without frameworks

Vaadin is a development platform for building web applications in Java. Although it includes a Java API (Vaadin Flow) that you can use to implement the User Interface (UI) entirely in Java without writing any HTML or JavaScript, the UI components included in Vaadin can be directly and individually used in any HTML document without having to use Vaadin Flow, Vaadin Fusion, or any other web framework.

Vaadin UI components are implemented as Web Components. In short, a Web Component is a custom HTML tag or element that a developer can define. For example, on YouTube, you can see that the developers created a custom element called ytd-comment-action-buttons-rendered to encapsulate the action buttons associated with a comment:

YouTube uses Web Components

Vaadin has a collection of modern Web Components, and in this article, you'll learn how to use them with the help of two tools: npm and Parcel. npm is needed to manage the dependencies. An example of a dependency is the Vaadin button Web Component (vaadin-button). This Web Component is implemented in a set of files that you need to download from somewhere. npm helps with that. Parcel is a zero-configuration JavaScript module bundler. It takes all the JavaScript files that make up your application plus the dependencies and creates a single JavaScript file that you can publish with your application when you deploy it to production.

Create a new project

Create a new directory for your application. For example:

> mkdir vaadin-components-without-frameworks

Download Node.js. You need it in since it contains npm. Install it and initialize a new project as follows:

> cd vaadin-components-without-frameworks
npm init -y

This creates a package.json file in the current directory. This file contains, among other things, the dependencies that your application needs.

Create a new JavaScript file with the name index.js. Create a new HTML file that loads the index.js file. Use index.html as the name of the file and create a simple document as follows:

<!DOCTYPE html>
<html lang="en">
<head>
    <title>Vaadin Web Components in HTML</title>
    <script src="index.js"></script>
</head>
<body>
    <h1>Using Vaadin Web Components without frameworks</h1>
</body>
</html>

Add Vaadin Web Components

Let's use Vaadin's button Web Component in this example. Download it using npm as follows:

> npm i @vaadin/vaadin-button --save

This creates a new node_modules/ directory where all the dependencies will reside. There, you'll find the files that form Vaadin's button Web Component and its dependencies. Web Components can have dependencies themselves.

Import the Vaadin's button dependency in the index.js file as follows:

import '@vaadin/vaadin-button/vaadin-button.js';

Using Vaadin Web Components

With the dependencies in place, add a Vaadin button in the index.html file as follows:

...
<body>
    <h1>Using Vaadin Web Components without frameworks</h1>
    <vaadin-button onclick='alert("It works!")'>Click me</vaadin-button>
</body>
...

Building and running the application

Download Parcel by running the following:

> npm install -g parcel-bundler 

To build the application, invoke Parcel as follows:

> parcel index.html

This creates the client-side bundle and the distribution files in the dist/ directory. These are the files that you can deploy to a web server in development, testing, or production environments. In fact, when you run the previous command, Parcel starts a development server. You can invoke the application at http://localhost:1234. Here's a screenshot:



Share:

Friday, August 13, 2021

New book - Practical Vaadin: Developing Web Applications in Java

I'm glad to announce the availability of my most recent book, Practical Vaadin: Developing Web Applications in Java published by Apress, a division of Springer Nature.

Practical Vaadin: Developing Web Applications in Java

The book teaches you how to develop Java web applications from scratch using the Vaadin open-source framework. You don't need to have any experience in web development to follow this book. On the contrary, the book starts by explaining the key technologies in web development and how Java fits into the picture. You'll learn how web servers operate, what a client/server architecture is, and how Servlets are the foundation of Java web development.

With a solid knowledge base established, you'll dig into web Graphical User Interface (GUI) creation using the Java Programming Language. The book shows how to add buttons, text fields, combo boxes, data grids, and how to connect the values in input fields to Java domain classes using data binding. You'll learn advanced topics such as authentication and authorization (login view and permissions), Server Push to update the UI from threads running on the server, the Vaadin's Element API to gain low-level access to the Document Object Model (DOM) in the browser, responsive design to automatically adapt the UI to different screen sizes, and much more.

The book also covers client-side UI implementation using Vaadin Fusion, a feature of Vaadin that allows you to implement views using the TypeScript programming language. It also covers integration with Spring Boot and Jakarta EE, alongside introductions to the basic concepts in these two technologies. The examples show how to efficiently connect to SQL databases using connection pools, and how to consume the data using the popular Jakarta Persistence API (JPA).

The book is available in print and digital formats on Amazon and many, many online bookstores. Here are some links:

I truly hope this book helps those starting their careers in web application development with Java, and those avid to learn more or refresh their knowledge on the powerful Vaadin Java web framework.

Share:

Friday, July 2, 2021

Designing an open-source Community Award

I had the pleasure to announce the Vaadin Community Award during Vaadin Dev Day Spring 2021. This award is a way to recognize members of the Vaadin Community who actively participate and help others through forums, social media, events, code, and more. The Vaadin Community has been growing rapidly over the past years and more and more individuals have become extremely active online and in conferences. In the Community Team at Vaadin, we figured it only makes sense to officially reward those who constantly help developers and decided to create that Vaadin Community Award (VCA).

I lead the effort by looking into other similar programs in the software industry. Companies such as Google, Oracle, Red Hat, Microsoft, and others have multiple programs to recognize the efforts of their community members. From ways to select the winners and promote the program to the program's name, we quickly made adjustments from the initial idea using the findings from others. For example, initially, we proposed the name Most Valuable Professional. However, the acronym didn't play well for a company that produces web frameworks—MVP is easily understood as Model View Presenter by web developers, or even worst, as Minimum Viable Product! From there, we proposed alternatives and selected the name Vaadin Community Award.

A brief description of what the program intends to achieve is always useful. So I wrote one that describes the essence of the program:

The Vaadin Community Award (VCA) is a program that recognizes community members with an exceptional track record on community participation. The VCA is given to members that actively lead online and local Vaadin communities, speak at conferences or Java User Groups, create and maintain Vaadin Directory add-ons, publish articles and videos, engage in social media, Discord, and Stack Overflow discussions, and in general, passionately help other members of the community.

This helps the team and the community understand what the program is about concretely. It helps to give examples of things that community members should do to win the award. I left the benefits out since this is better explained with bullet points, and it shouldn't be (and it usually is not) the main motivator for members to be actively engaged in the community. Often, the driving factor is to get recognition and experience to show that they are proficient or have the passion for working on technical projects or communities around open-source software.

Coming up with a set of benefits is important nevertheless. From officially receiving the recognition to getting tangible prizes, a good set of benefits have the potential to keep the winners engaged in the community and inspire others to join the selected group. At first, the idea was to give a certificate that the winners can download plus extra powers in the Vaadin Discord server (chat). This combination works well as it officially recognizes the title and gives more tools to the winner to continue to help the community. We later added a physical recognition token that we would send to the winners, like a trophy that they can put in their homes to remind them how valuable they are to the Vaadin Community. We also decided to create a web page with information about the program and a list of winners to boost the recognition part. I proposed to give them a one-year subscription to the Pro offer of Vaadin for free—if someone has been helping the community to the point of earning this award, they should definitively have all the tools Vaadin develops in their belt. And to complete the package, we added the possibility to be interviewed for the Community Spotlight series.

Another aspect to consider when designing a community award is how to select the winners. Who nominates members, and who picks the winners? How many of them and for how long do they retain the title? For the VCA program, we decided that previous winners should nominate new candidates and the whole community select the winners through voting. By doing this, Vaadin yields control of the program to the community. We initially decided to give the award for a period of one year, allowing winners to be nominated and win the award in subsequent years. The goal was to encourage previous winners to continue to work for the community. However, if previous winners are the ones nominating new ones, this wouldn't make much sense, so we discarded the idea. When the program starts, there are no winners yet, so we decided to select the initial ones internally at Vaadin to get a good starting team.

In summary, come up with a good name and description, select a good set of benefits, and come up with a good way to select the winners avoiding situations like "you have to do this specific thing to win." Let the community participate in the process as much as possible.

Share:

Friday, April 30, 2021

How to participate in and contribute to open source projects

I like to think about open-source involvement in terms of levels. Although no level is more important than the other, they build up on top of each other and take you through a good path to become a key member of an open-source community. In short, the levels are:

  1. Observer
  2. Member
  3. Contributor
  4. Committer
  5. Maintainer

Become a user of the product

The best way to get inspired and start contributing to an open-source project is by using it. It's a natural first step. You'll learn about the vocabulary around the product, its strengths, and its weaknesses from a user perspective. Your interest in improving the product will increase alongside your motivation to contribute back to the community.

This level makes you an observer.

Join the community

Once you become a regular user of the software, you'll feel comfortable using the software and talking about it. You'll learn where to find help and how others use the software. You'll visit community forums and start asking questions. Eventually, you'll also answer questions. You are participating in the open-source community at this point. Forums are not the only way to participate in the community. You can present the software to others, talk about it, and promote it with your peers. The critical part here is that you interact with others who are or might be interested in the software.

This level makes you a community member.

Report bugs, issues, and improvements

Once you become a regular user, you can start reporting defects (bugs, issues) to the maintainers. Open-source projects typically have online tools for this. For example, if the project is hosted on GitHub, you'll most likely find an issues tab. Be considerate, and check if the bug has already been reported. If that's the case, see if you have new information useful to the maintainers. If not, open a new case and provide valuable information on the issue. If they have any, follow the guidelines of the project to report issues. Make sure you use formatting tools (especially for logs or source code when applicable) and attach relevant screenshots if they are relevant. Suggest improvements on issues but also as new ideas.

This level makes you a contributor.

Send source code contributions

If you are a software developer, this is the level you are most interested in. Your path towards contributing code can start with simple patches, add-ons, or documentation. They are frequently managed in the same way—through source code version control systems such as Git. Look for contribution guides in the project website, download the source code, and set up a development environment. Some projects are massive in terms of the number of lines of code, so be prepared to spend some time getting familiar with the big picture and focus on building and running the project. I guarantee you that seeing the software running from your very own built is a highly satisfying and motivating experience. Experiment by changing the code and seeing the results. Jump to the project's bug tracker and see if there are any easy-to-tackle bugs you could hunt. Often, maintainers mark a bug as a "good first issue" when they estimate that the effort required to fix it is low. Always go through the contribution guidelines for committers and stick to the processes that the project has established for good collaboration.

This level makes you a committer.

Become a key member of the project

As you contribute more and more to the project, you'll find that other community members start to recognize and trust your work. You have demonstrated your skills and passion for the project, and people trust your ability to make good decisions to help the project succeed. In many open-source projects, maintainers can nominate or even directly select community members to become permanent contributors to the source code. You'll start reviewing other's patches and, likely, implementing new features and help drive the future of the project. Only genuine interest and constant work in the project will take you there. The road can be long, but I promise, it's an exciting one at each of the levels I describe here.

This level makes you a maintainer.

Share:

Sunday, April 18, 2021

Blog rebrand

Hey reader! A short entry to announce and document my blog's rebrand. These are the main changes: 
It's been a while since I wrote for my blog. I had been busy publishing mostly on vaadin.com and dzone.com, but I do want to keep this one alive and this rebrand is proof of my motivation. I wanted to move the focus to the content rather than the author's profile. So, I found a Blogger theme that fits this purpose well. It doesn't show lateral menus all full of me, me, and me. Instead, most of the space is organized around the articles without distracting images and with the important information highlighted. At the bottom of the pages, you can always find a footer with more information about the website.

From now on, I'll try to share useful information around programming, software development, Java and its ecosystem, Vaadin, databases, and other technical topics. I also want to share thoughts on Developer Relations (or Developer Advocacy) and open-source community management. You can always send suggestions on topics that you would like me to discuss as well. I'd love to hear from you!

Big thanks to Sandpatrol for the design, NewBloggerThemes.com for the theme, and Blogger for the hosting.
Share: