I. Healthcare Platform Requirements

1. Scenario

General Hospital A has been providing City X‘s citizens with a wide range of healthcare services. Its routine operations rest on its Data Center on-premises with a legacy healthcare platform composed of heterogeneous monolithic, SOA apps that were designed and built up 2 decades ago. The growing population results in the high demand on the high load, high quality and multi-specialties of healthcare services.

With the as-is legacy healthcare platform, its day-to-day operations undergo a series of pain-points, incidents, and limitations including inflexible business functionalities, slow response time under high load (even saturation), API management, resilience, observability, maintainability, expandability, deployability… To address these shortcomings of the as-is platform, the hospital hires a multi-national IT consulting firm who advises the Hospital Leader Board to modernize their as-is healthcare platform by capitalizing on advantages and strengths of cloud computing, Microservices as a trendy architectural style, Intelligent Analytics. As a SA, you are tasked with architecting a MSA-based healthcare platform for Hospital Leader Board and development teams with requirements below.

2. Business Specification

The healthcare platform manages information about patients (demographics, medical history, insurance data). It also manages Electronic Medical Records (EMR) including medical tests (blood, urine...), imaging studies (X-ray, CT/MRI scans, Ultrasound...). It allows patients to online schedule appointments with doctors in advance. Patients can follow the normal medical procedure by online booking doctors, or offline registering for examinations, paying service bills in advance. Receptionists receive patients' requests and arrange appointments with doctors. On appointment days, patients go into the hospital, show up at Examination Department, doctors examine patients, request medical tests, if necessary. Based on symptoms, test results, doctor will diagnose diseases, decide on treatments (surgery, medication, therapies), prescribe medicine. Patients pay bills for additional service fees, if any, deposit money in case they undergo inpatient treatments and monitoring. Patients can also claim insurance if available. The healthcare provider provides emergency, check-up services, inpatient and outpatient treatment services to the City's citizens.

3. Technical Specificaiton

The platform integrates with External Identity Provider, External Payment Gateway, External Insurer, Internal Accounting and Logistic systems. It supports both web browsers and mobiles as clients.

II. Requirement Analysis

We need to design a new MSA-based healthcare platform to replace the legacy system (on-premises, monolithic + SOA):

• Capable of handling thousands of concurrent users

• Capitalizing on advantages and strengths of cloud computing, microservices and cloud app best practices.

• Implement data replication and data recovery strategies to prevent data loss

• Utilize distributed systems principles to ensure high availability and fault tolerance

• Maintain data consistency across microservices

Therefore, we need to decompose the Healthcare Platform as the whole business domain into corresponding subdomains/bounded contexts.

Functional Requirements

• Manages:

  • Patient information (demographics, medical history, insurance data)

  • Electronic Medical Records (EMR) including medical tests (blood, urine...), imaging studies (X-ray, CT/MRI scans, Ultrasound...)

• Allows patients to online schedule appointments with doctors in advance

• Patients can pay service bills, claim insurance

• Receptionists receive patients' requests, arrange appointments with doctors

• Receptionists/Doctors handle medical procedures, store medical history,... into the system

• Provides emergency, check-up services, inpatient and outpatient treatment services to the City's citizens.

• Integrates with External services (Identity Provider, Payment Gateway, Insurer), Internal services (Accounting, Logistic, Resource Management).

  

Notes: We must clarify the scope of the new platform with customers. For instance, Resource Management, Pharmacy Management might not be explicitly mentioned in the requirements, but it's important to confirm whether it should be included.

Non-functional Requirements

• Capable of handling thousands of concurrent users

• Data replication and data recovery

• High availability with minimal latency

• The system should be scalable and efficient

• Data encryption for all layers, end-to-end encryption

III. Subdomain Decomposition

I often apply the 4+1 Architectural View Model for my architectural designs, providing comprehensive perspectives for stakeholders with 5 different views. I'll share details about this model and complete designs for our healthcare platform in another article. In this article, I'll focus only on the Logical View, including our subdomain decomposition process. Btw, I will skip Ubiquitous Language table creation since I'm the only one working on the project 😂

1. EventStorming Workshop

To clearly decompose the entire domain, we first need to review the main business process:

I simulated an EventStorming workshop and identified all key events and created a following Big Picture:

2. Identify Bounded Contexts

From the above Big Picture, I can identify natural boundaries between subdomains.

Finally, I categorized subdomains into the following three groups:

Core Subdomains

• Treatment Service: manages medical process during patient’s examination, inpatient & outpatient treatments

• EMR (Electronic Medical Records): manages Electronic Medical Records (EMR) including medical tests (blood, urine...), imaging studies (X-ray, CT/MRI scans, Ultrasound...)

• Patient Management: manages patient information, medical check history,…

• Emergency Services: emergency and urgent care services

• Check-up Services: handle check-up processes, check-up services for personal/enterprises

Supporting Subdomains

• Reception Service: manages patient reception, registration, guidance

• Appointment Scheduling: manages appointments with doctors (online and offline registration)

• Billing & Payment: handles payments and bills/invoices for medical services

• Insurance Management: handles patients claims, manages insurance information

Generic Subdomains

• Account management: handle the authentication/authorization with external Identity Provider, manage accounts, roles,…

• Notification: handles notification services (SMS, email, mobile notifications)

• Reporting/Analytics: manages/generates reports and dashboards for monitoring and data-driven decisions.

3. Identify Context Map

As I mentioned in the previous post, I only use upstream/downstream relationship for Context Map to keep it simple and more readable. Integration patterns like Anti-Corruption Layer (ACL), Open Host Service (OHS),… with more details might be defined in the architecture design.

As this is our example, I didn’t have any workshops with business team (very important) but relied on my knowledge of healthcare systems and my experience with the My HMUH mobile app.

In complex domains, it can be challenging to capture every relationship between subdomains on a single Context Map. Therefore, it's a good idea to split it into multiple maps.

Importantly, building Context Map will reveal any conflicts or redundancies in the business flow, helping to refine the design before proceeding to detailed development.

IV. Conclusion

Phew… We've went through an interesting example of applying DDD to a healthcare platform design, exploring how to break down complex medical processes into subdomains and identify Context Map between them. It took me a lot of time to complete the DDD series.

Once again, I hope this helps everyone get a good start with DDD and be ready to handle larger projects. If you find it useful, feel free to share.

References

• Mr Dung Ho - Presales Consultants, Solution Architect - FPT Software

Over the years, as business needs have grown increasingly complex, our application systems have evolved - from monoliths to SOA, and now to microservices. This evolution demands a rational approach to component decomposition and integration.

Many companies, including the one where I previously worked, have experienced systems ballooning over time due to improper decomposition, leading to tightly coupled data and components. This often leads to problems such as domain leaks, inconsistent data, overly large codebases, and frequent system failures.

Source: https://medium.com/startlovingyourself

Domain-Driven Design (DDD) emerges as a systematic approach for large-scale projects with complex domains. By promoting clear boundaries between subdomains and a shared language between business and technical teams, DDD helps to avoid the pitfalls of past systems. It ensures every component is self-contained, with clearly defined responsibilities that accurately represent the specific business processes they support.

II. What is DDD?

Domain Driven Design (DDD) is a development philosophy defined by Eric Evans in his seminal work Domain‐Driven Design: Tackling Complexity in the Heart of Software (Addison‐Wesley Professional, 2003).

DDD is an approach to software development that enables teams to effectively manage the construction and maintenance of software for complex problem domains. The main idea is splitting out the whole business domain into subdomains and microservices correspond to these subdomains.

For example, in designing a logistics system, you might divide the domain into subdomains such as Order, Warehouse management, Shipment, Routing, Address,…

Importantly, DDD can be applied to both monolithic and microservices architectures.

III. Step-by-Step DDD Process

DDD is very difficult to start, especially for newcomers. The DDD books are quite academic and verbose, which can be overwhelming. Therefore, I have summarized it into a 5-step process to help those new to DDD practice it as easily as possible. This guide takes you from understanding the requirements all the way to being ready for coding.

1. Identify problem space and business domain

1.1. Do assessment

Start by asking

• What business problem am I trying to solve?

• Who are the primary users of the system?

• Is this project a new system or a legacy migration?

• What parts of the current system should be preserved, and what needs to be rebuilt?

• Which parties need to be integrated?

• What pain points exist with the current system?

• What is the project’s goals, timeline and budget?

• …

This is a very important step. A wrong scope or a misunderstanding of resource constraints can lead to many consequences - redesign, rework, missed deadlines - and can also diminish the organization's credibility with its customers (especially in outsourcing).

Let's clarify the goal, timeline, and budget very carefully with the customers or your boss, because you will need to adjust the design to fit these constraints.

1.2. Talk to the domain experts/BA

We must involve people who know the business well - Product Owners, BA, managers - to get real insights and confirmation. Remember: Never Assume!

2. Build a Ubiquitous Language

Ubiquitous Language is a shared vocabulary between domain experts and developers. It ensures every term used in the code, documentation, and conversations accurately reflects business concepts.

Example (Online Store System):

Terms like Product, SKU, Category, Cart, Order, Checkout, Support Ticket,… should have precise definitions, ensuring clear communication between domain experts and developers.

Best Practices for Shaping the Language

• At the beginning of a project, we should create a dictionary of domain terms.

• Ensure that you use one word for a specific concept. Don’t let the domain expert or developers have two words for something.

• If you are using a term in code that the domain expert doesn’t say, you need to check it with them.

• The business may use some terms proving to be too generic. The development team and domain experts need to create new terms and explicitly define the meaning of existing terms to implement the model in code.

3. Subdomain Decomposition

There are two common decomposition methods: by business capabilities and by Event Storming workshop.

3.1. Business Capability Decomposition

A business capability is a service (without the aid of technologies) that generates business values to a(n) business/company/organization as well as end users.

For example, the business capabilities of an online store include:

• Product catalog management

• Inventory management

• Order management

• Delivery management

• Customer Service

To identify business capabilities, we should focus on the organization’s purpose, structure, and processes. Different groups/departments within an organization might correspond to business capabilities or business capability groups.

This method is especially useful for large companies with clear, established organizational structures.

3.2. Event Storming Workshop

This approach begins by running an Event Storming workshop to map out all the key events, commands, and processes from start to finish. You can watch this video guide to learn more about Event Storming in detail. In short, there are three workshop types in EventStorming:

• Big Picture: visualizes the entire business landscape by identifying key events, actors and interactions to establish a shared, high-level understanding

• Process Modeling: dive into specific business flows that are separated from Big Picture

• Software Design: transform events from Process Modeling into concrete software designs (Aggregate, Entity, Value Object,…)

There are also a few main concepts that you need to remember:

To decompose subdomains, we only need to create a Big Picture. Below is a sample Big Picture for our online store system, where I identified all domain events arranged sequentially from start to finish.

By visualizing the business flow end to end, you not only gain a deeper understanding of how the domain operates, but you also reveal natural boundaries between subdomains. Once these boundaries become clear, you can decompose the system into distinct subdomains that reflect different parts of the business process.

After identifying these subdomains through Event Storming, you can classify them into three main types:

• Core - key differentiator for the business and the most valuable part of the application.

• Supporting - related to what the business does but not a differentiator. These can be implemented in-house or outsourced.

• Generic - not specific to the business and are ideally implemented using third-party solutions.

For example, the subdomains of an online store include:

• Core: Product Catalog, Order Management, Customer Service

  → they offer differentiates your business in the market and becomes a key driver of customer loyalty and competitive advantage.

• Supporting: Inventory Management, Payment, Promotion

  → they are essential to operations but do not, by themselves, provide a unique competitive edge.

• Generic: Delivery Management, Notification,…

  → many businesses can rely on established third-party logistics solutions for delivery.

This method is ideal for smaller organizations that don’t have rigid business structures but can be applied to any organization to any extent.

4. Identify Context Map

After identifying subdomains, we build Context Mapping - represents the relationships among bounded contexts (upstream, downstream, dependencies…).

Source: contextmapper.org/docs/examples

There are four common patterns:

• Upstream/Downstream Relationship

  

• Anticorruption Layer: a layer for integration, only contains translation logic, not business logic. We use this layer to integrate with code we don’t own or can’t change.

  

• Shared Kernel: uses a shared model for 2 bounded contexts (have a lot of crossover in terms of domain concepts and logic). 2 bounded contexts can be in the same subdomain

  

• Open Host Service: reduce duplication of ACL by exposing an API to be reused by many contexts.

  

However, after trying to draw context maps based on these patterns, I found them to be quite complex and hard to understand - especially as the number of bounded contexts increases. So, I simplified the approach by using just one type of upstream/downstream relationship, but divided it into two categories:

• Synchronous integration via API

• Asynchronous integration via events

By this way, everyone attending the workshop, especially the business team, can understand it easily.

5. Identify DDD Artifacts

Below are the key domain concepts that serve as the fundamental building blocks for our design after subdomain decomposition.

• Aggregate: a group of related objects and treated as a single unit. It has an aggregate root that controls access to other objects. Only the root can be referenced from outside the aggregate

• Aggregate Root: a central entity - entry point for a aggregate, all other entities are accessed only via root

• Entity: a domain object that have a distinct identity

• Value Object: an object that contains data but don't have an identity

• Domain Event: An event that captures a significant change within the domain and notifies other components of the system

• Domain rules: business logic and constraints that govern the behavior of the domain

• Domain service: services/module encapsulate the domain logic that does not naturally fit within an entity/aggregate. Domain services use entities, value objects, or other domain services to execute domain-specific operations

Additionally, you can define domain operations (Queries and Commands) and long-running transactions (Sagas) that coordinate processes spanning multiple aggregates.

Example: Order Management Subdomain

• Aggregate root: Order

• Value Object: OrderItem, ShippingAddress, PaymentDetails,….

• Domain Event: OrderPlaced, OrderCancelled, OrderDelivered,…

• Domain Service: OrderValidationService (validate that the order details, check for inventory availability,… that does not fit within only Order Aggregate).

IV. Pros and Cons of DDD

DDD is not a silver bullet. Below are its benefits and drawbacks to help determine when it’s the right approach:

Advantages

• Enables teams to manage complex problem domains using a divide-and-conquer approach

• Encourages close collaboration between technical team and domain experts to ensure the software reflects real-world business processes

• Establishes clear domain boundaries, making it easier to update or replace system components without affecting others

• Avoid domain leaks between components, especially in microservices

Disadvantages

• Can be complex for developers to fully understand and implement

• Slower initial progress

• Requires continuous refinement of the domain model

• Offers less value for small or CRUD projects

Use cases

• Large enterprise systems with complex domains such as banking, insurance, healthcare, or logistics,…

• Microservices Architectures where each service aligns with a specific subdomain or business capability

V. Conclusion

Phew… We've covered the key concepts of DDD along with best practices for implementation. There's so much I'd like to include in this article, but I'm concerned it might be overwhelming. Therefore, I've focused on introducing the basic concepts and best practices, leaving additional details for you to explore if you're interested.

Once again, I hope this helps everyone get a good start with DDD and be ready to handle larger projects.

VI. References

• Mr Dung Ho - Presales Consultants, Solution Architect - FPT Software

• Patterns, Principles, and Practices of Domain-Driven Design - Scott Millett, Nick Tune

• Domain-Driven Design: Tackling Complexity in the Heart of Software - Eric Evans

I. A Bigger Picture

Over the past five years, CI/CD has become a fundamental practice for modern software engineers.

According to Martin Fowler:

Continuous Integration is a software development practice where each member of a team merges their changes into a codebase together with their colleagues changes at least daily. Each of these integrations is verified by an automated build (including test) to detect integration errors as quickly as possible.

Yes, as quickly as possible. You could rely on manual testing, but it’s slow, prone to human error. To fully support Continuous Integration, automated testing is essential - something that runs and verifies your code every time you make a change.

Automated tests act as an early warning system, catching bugs before they reach production. With a solid suite of tests in place, you gain confidence that your changes won’t break existing functionality.

II. Testing Methodology

In automated testing, we commonly categorize tests into three main types:

• Unit Tests – Focus on testing individual components in isolation.

• Integration Tests – Verify interactions between multiple components or services.

• End-to-End (E2E) Tests – Simulate real user flows to ensure the entire system works as expected.

The test pyramid provides a clear guideline on how to distribute tests across different layers. At the base, you should write a lot of small, fast unit tests. In the middle, you have integration tests, which are fewer but validate interactions between components. At the top, you write only a few high-level end-to-end (E2E) tests, which ensure the system works as a whole.

Lower levels focus on isolated code behaviors, while higher levels validate component integration and overall system functionality.

In this article, I’ll focus on the foundation of the pyramid - unit tests and explore what makes them effective.

III. What is a unit test?

Definition

A unit test is an automated test that:

• Verifies a small piece of code (also known as a unit, often a method)

• Runs quickly

• Executes in isolation from dependencies or external systems as databases, APIs,…

Structure

A unit test follows a simple but effective pattern, often called the AAA pattern:

• Arrange – prepare the dependencies and data of a scenario

• Act – Call the testing method and capture the return value (if any)

• Assert – Verify the outcome

And remember: never write a test without an assertion! Unless, of course, you just want to beautify your test report. 😆

Naming

Proper naming helps you understand what the test verifies. A good name encourages you to focus on the behavior of the code instead of the implementation details.

You can also use a very common convention is: MethodUnderTest_Scenario_ExpectedBehavior

Test Doubles

A test double is a term that describes fake dependencies in tests. There are five variations of test doubles: dummy, stub, spy, mock, and fake.

• Mock: Emulates and verifies outgoing interactions - checking whether a method was called, how many times, and with what parameters. These interactions are calls the SUT (System under test) makes to its dependencies to change their state (e.g., sending an email, calling external services).

  

• Stub: Emulates incoming interactions by providing predefined responses/exceptions. These interactions involve the SUT calling its dependencies to retrieve input data (e.g., fetching data from a database or external system).

  

• Spy: Partially mocks an object while keeping some real behavior intact.

  

• Dummy: A simple, hardcoded value (e.g., null or a placeholder string) used only to satisfy method parameters but never actually utilized.

  

• Fake: A lightweight implementation of a dependency, such as an in-memory database or HashMap, used as a replacement for a real system (e.g., replacing an actual database)

• 

IV. What makes a good test?

1. Protection against bugs

A test that shows good protection against regression depends on:

a. Amount of code executed during the test

Generally, the larger the amount of code that gets executed, the higher the chance that the test will reveal a bug.

b. Complexity and domain significance

Code that represents complex business logic is more important than boilerplate code. Therefore, our tests should focus on core domain components.

2. Fast feedback

A good unit test provides fast feedback, allowing developers to catch issues early in the development cycle. When a test fails immediately after a bug is introduced, fixing it is quick and inexpensive.

The cost increases exponentially when the bug is not discovered early. A fast and reliable test suite helps maintain development speed while preventing costly bugs.

3. Maintainability

• Keep your unit tests small and readable.

• A single test should not take a lot to arrange and too many assertions.

• Independent of out-of-process dependencies: databases, APIs, message queues,…

4. Resistance to refactoring

If a test breaks due to refactoring of underlying code without any change in behavior, it indicates poor test design.

Example: Scooter Configuration

Imagine you want to buy a scooter with:
✅ A blue handle
✅ Illuminating wheels
✅ A red bell

The constructor receives custom settings to configure the scooter, then configuration() method publishes the configuration for user to see.

Now, consider the following unit test:

• It initializes the Scooter class with custom settings.

• It asserts that the Scooter contains:

  • The first instance of the Handle class.

  • The second instance of the Wheel class.

  • The third instance of the Bell class.

The Problem

However, the test is looking at the implementation details of the class. It will fail if

• We alter the order of part initialization to configure Wheel first, instead of Handle

• We refactor the code to say that Bell is a sub-part of Handle

The Solution

Instead of asserting the exact instance positions, a refactoring-resistant test should verify:
✅ The Scooter and Bell has a handle with correct color.
✅ The Wheels have the expected illumination feature.

A better test focuses on observable behavior. Refactoring the underlying code like moving around the objects, or changing the composition won’t impact the test.

Where to find observable behavior?

If you are working with Clean Architecture, the observable behavior should reside in Domain and Use case layers, where the core business logic is implemented.

If you are working with Controller-Service-Repository projects, the observable behavior should reside in the Service layers.

V. Common Mistakes in Unit Testing

1. No assertions

A unit test without assertions is not really a test - it’s more like “just writing code for coverage”. A unit test should always assert that the code behaves as expected. If your test doesn’t verify the result, you’re not testing anything at all.

2. Test private methods

One of the most commonly asked questions is how to test a private method. The short answer: you shouldn’t.

Testing private methods violates an important principle mentioned earlier: test only observable behavior. Exposing private methods couples tests to implementation details, making them fragile and reducing their resistance to refactoring.

However, talk is cheap. Sometimes the codebase is just too bad 😆 to strictly follow this principle. That’s why PowerMock exists - it allows developers to mock private methods and more.

3. Too large Arrangement

A large arrangement section in a unit test can be a sign of problems. If your setup code is lengthy and repetitive, it makes tests harder to read and maintain.

When the arrangement code is large and repeated across multiple tests, consider refactoring:

• Use helper methods to set up commonly used objects or scenarios with parameters.

• Use factories to create reusable setups.

• Consider mocking or using a framework like Mockito to simplify the setup.

4. UT contains implementation details

If you need to invoke more than one method in your act statements:

Please encapsulate implementation details in your code, don't couple them with unit tests.

5. Multiple act statements in a UT

We should write one unit test per scenario, meaning each unit test should have only a single act statement.

Avoid writing tests with multiple Act statements, as shown below, as it makes the test unclear and harder to understand the scenario being tested.

6. Trivial test

Attribute test

Instance test

Such tests are insignificant to the system and lower resistance to refactoring.

7. Conditional test

Separate conditions into different unit tests instead of using conditional statements within a single test.

8. Stub test

There is no value if we try asserting the stub data that we emulated in the Arrange phase.

9. Leaking domain knowledge

Don’t let your tests hold domain knowledge. Tests should focus on verifying behavior.

VI. Conclusion

Phewwww! We've just covered some basic concepts and practices. I hope they help you get started with unit testing and make your software more reliable.

This is an interesting topic and might spark some controversial discussions. If you have any comments, feel free to share!

Reference

• Unit Testing: Principles, Practices, and Patterns

Introduction

Definition of Hexagonal Architecture

An architectural pattern which creating loosely coupled application components that can be easily connected to their software environment by means of ports and adapters. This makes components exchangeable at any level.

Hexagonal vs Clean Architecture

Hexagonal or Clean architecture have the same objective, which is the separation of concerns.

• Divide the software into layers

• Independent of frameworks, UI, database, external services

• Testable without UI, database, web server, or any other external services

You can see the similarity between them.

Core Concepts of Hexagonal Architecture

Ports

Port is an interface layer that define how core domain layer interact with external components (frameworks, external services, databases,…).

• Inbound ports: handle input to the core, such as user commands or requests from clients or other systems.

• Outbound ports: define how the core can communicate with external systems like databases or APIs.

Adapters

Adapters are the implementations of ports.

• Inbound Adapters: These include things like REST controllers, CLI interfaces, or event-driven consumers that handle user input or external events.

• Outbound Adapters: These are responsible for communicating with external systems, such as databases, third-party services, or message brokers.

Domain Model

This is the core domain layer where holds domain entities and business logic. It’s isolated from frameworks and external components and is the most stable part of the software.

Application Services

An application service acts as a facade through which clients interact with the domain model. This is where ports are defined.

It control database transactions, orchestrates business operations but should not make any business decisions (should be in domain layer).

Setting Up a Java Spring Boot Project

Prerequisites and Tools Needed

• Java 17

• Spring Boot 3

• Intelij or any Java IDE

Initializing the Project

We start with https://start.spring.io/ to create a new Spring Boot framework. I chose Springweb and Lombok dependencies only for the demo.

You can check the project structure on the website before downloading it to your local.

Implementing Hexagonal Architecture with Java Spring Boot

I will implement a booking service where we can create a booking as a sample.

To implement 3 layers of the architecture, I created 3 Java modules:

• Domain: store domain entities and business logic

• Application: store inbound and outbound ports

• Tech Framework: includes frameworks and adapters for database, controllers, external services,…

In which, Application module depends on Domain module:

and Tech Framework module depends on both Application and Domain:

As you can see, the domain has no dependency on frameworks or external components.

Developing the Domain Layer

Firstly, I have a BookingDomainEntity (just a sample, ignore the attributes pls :D)

import java.time.LocalDateTime;
import lombok.*;

@Getter
@Setter
@Builder
@AllArgsConstructor
@ToString
public class BookingDomainEntity {
  private long id;
  private long userId;
  private long totalAmount;
  private String pickUpAddress;
  private String dropOffAddress;
  private LocalDateTime createdOn;
}

Secondly, I have commands for business logic. To keep it simple, I don’t use command for booking creation.

Developing the Application Layer

As I showed you, we have inbound and outbound ports in this layer.

To creat a booking, I have a BookingCommandInboundPort so that it can be called by controllers from Tech Framework layer.

@Slf4j
public class BookingCommandInboundPort {

  private final BookingRepositoryOutboundPort bookingRepositoryOutboundPort;

  public BookingCommandInboundPort(BookingRepositoryOutboundPort bookingRepositoryOutboundPort) {
    this.bookingRepositoryOutboundPort = bookingRepositoryOutboundPort;
  }

  public void save(BookingDomainEntity entity) {
    log.info("Saving BookingDomainEntity");
    bookingRepositoryOutboundPort.save(entity);
  }
}

And a BookingRepositoryOutboundPort for saving a new booking to database:

public interface BookingRepositoryOutboundPort {

  List<BookingDomainEntity> findAll();

  void save(BookingDomainEntity entity);
}

Then the application service should look like this:

Designing the Tech Framework Layer

Finally, we implement the outsidemost layer. The dependencies that I chose previously is for this layer. I also added mapstruct library for object mapping.

We also have inbound and outbound structure similar to the application layer.

Firstly, I have BookingEntity and BookingRepository for persistence. To keep it simple, I don’t use database and only create a dummy entity and repository:

// should have @Entity in real project
@Data
@Builder
public class BookingEntity {
  private long id;
  private long userId;
  private long totalAmount;
  private String pickUpAddress;
  private String dropOffAddress;
  private LocalDateTime createdOn;
}

@Repository // Dummy repository => should be an interface and extend JpaRepository
public class BookingRepository {

  List<BookingEntity> findAll() {
    return List.of(BookingEntity.builder().build());
  }

  BookingEntity save(BookingEntity entity) {
    // dummy save method
    return entity;
  }
}

Secondly, I have a BookingRepositoryAdapter that implement BookingRepositoryOutboundPort (from application layer):

@Component
@RequiredArgsConstructor
public class BookingRepositoryAdapter implements BookingRepositoryOutboundPort {

  private final BookingRepository bookingRepository;
  private final BookingMapper mapper;

  @Override
  public List<BookingDomainEntity> findAll() {
    List<BookingEntity> entities = bookingRepository.findAll();
    return mapper.toBookingDomainEntity(entities);
  }

  @Override
  public void save(BookingDomainEntity entity) {
    // Transform Aggregate Root 's states into JPA Entity 's states before saving them into database
    bookingRepository.save(mapper.toBookingJPAEntity(entity));
  }
}

and a BookingMapper for object mapping. I strongly recommend you try Mapstruct to eliminate these mapping boilerplate code.

@Mapper(componentModel = "spring")
public interface BookingMapper {

  @Mapping(target = "createdOn", ignore = true)
  BookingDomainEntity toBookingDomainEntity(BookingRequest request);

  BookingEntity toBookingJPAEntity(BookingDomainEntity entity);

  List<BookingDomainEntity> toBookingDomainEntity(List<BookingEntity> entities);
}

Ok finally, we need a BookingController to handle client requests:

@RestController
@RequestMapping("/api/v1/bookings")
@RequiredArgsConstructor
public class BookingController {

  // Inbound port in application layer
  private final BookingCommandInboundPort bookingCommandInboundPort;

  private final BookingMapper mapper;

  @PostMapping(
      consumes = {MediaType.APPLICATION_JSON_VALUE},
      produces = {MediaType.APPLICATION_JSON_VALUE})
  public ResponseEntity<BookingResponse> createBooking(@RequestBody BookingRequest request) {
    bookingCommandInboundPort.save(mapper.toBookingDomainEntity(request));
    return ResponseEntity.ok().build();
  }
}

If you are feeling lost in this architecture, you are not alone :D. The flow is quite complex and can be confusing. Don't worry, I’ve sketched out a request flow so you can better visualize the entire process we just discussed:

From this perspective, it's clear that if I want to replace my Spring Boot with Quarkus in the future, I can retain the application and domain modules since they are not framework-dependent. This will save a lot of effort, trust me! 😊

You can check the full code here.

Conclusion

We've just covered the definition and a hands-on example of Hexagonal Architecture. You might not fully understand it after reading, but I recommend trying out the sample first. Then, when you come back, I believe things will be clearer for you.

Next time, I will implement a DDD sample in this repo and we will discuss about why we should apply Hexagonal along with DDD for our project.

See yaa!!!

References

• https://en.wikipedia.org/wiki/Hexagonal_architecture_(software)

• https://vaadin.com/blog/ddd-part-3-domain-driven-design-and-the-hexagonal-architecture

• https://www.happycoders.eu/software-craftsmanship/hexagonal-architecture/

Please bear in mind that you must practice these strategies to master them. Let's create your database and try with sample data such as Sakila,...

I. Choose The Right Column

Firstly, please list all queries for that table before deciding which indexes to create.

Secondly, don't create the index for columns with less distinct values (low cardinality).

E.g. status, action, is_active,...

You can check the cardinality figures by SHOW INDEX FROM + your tables

However, if the distribution of your ‘status’ column is unbalanced => Index still works if you only search by a value with lower distribution.

II. Create Prefix Index

If you need to create an index for a string/varchar column, I believe that you should create a prefix index:

• Long enough to work efficiently

• Short enough to reduce the index's storage

Example

Let's see the customer table with an index of the last_name column here and calculate the difference between the full-length index and prefix index of the first 6 chars:

• Full length: key length = 45 * 4 + 1 (store length) = 181 bytes

• Prefix index: key length = 6 * 4 + 1 (store length) = 25 bytes

Yes, the key length is 7 times larger.

How to find a good length

A good way to calculate a good prefix length is by computing the full column’s selectivity and trying to make the prefix’s selectivity close to that value.

E.g.: I have a city_demo table and I want to create an index for the city column.

• Step 1: we target a selectivity near 0.0312 as full-length selectivity.

• Step 2: evaluate many different lengths in one query, which is useful on very large tables:

• Step 3: This query shows that seven characters are good enough for my prefix index.

III. Composite Index

Sometimes, you need to query by 2 columns in the same query. Please consider creating a composite index instead of an index for each column.

Individual indexes on lots of columns won’t help our database improve performance for most queries.

For example, MySQL can cope a little with such poorly indexed tables when it employs a strategy known as index merge, which permits a query to make limited use of multiple indexes from a single table to locate desired rows. It can use both indexes, scanning them simultaneously and merging the results.

However, the algorithm's buffering, sorting, and merging operations use lots of CPU and memory resources. So a composite index should be your first choice.

Another point is When you create a composite index for multiple columns, you can use that index in a query with the first column of the index.

E.g: The index of (username, created_at) can be used for both queries:

• Select * from users where username=‘hieupd’ and create_at > ‘2022-01-01’

• Select * from users where username=‘hieupd’

\==> No need an index for the username column only.

IV. Choose The Order for Composite Index

How?

Did you decide to create a composite index? OK good!

Now how about the order of columns?

I asked the question to at least 30 interviewees, and only one of them could answer.

The answer is:

Choose the column with higher selectivity as the first column (in most cases)

We can check the selectivity of columns by the query count distinct / count *.

Pay Attention!!!

• When creating an index (A, B), no need to create an index only for A

• Should put your range condition to the right of queries

  In B-tree, if the range condition comes first in WHERE, the second column will not be used in that index

  E.g.: In the above image, I have a composite B-tree index of (date_of_birth, subsidiary_id)

  If I query by:
  Select * from uses when date_of_birth between ’01-Jan-71’ and ’09-Jan-71’ and subsidiary_id = 27;

  \=> Our database only uses the index for the first range condition, and then scans on left nodes for the second condition (subsidiary_id = 27) without index supports.

V. Clustered Index

Definition

Clustered indexes aren’t a separate type of index. Rather, they’re an approach to data storage.

When a table has a clustered index, its rows are actually stored in the index’s leaf pages.

• The leaf pages contain full rows, but the node pages contain only the indexed columns.

• You can have only one clustered index per table because you can’t store the rows in two places at once.

• Clustered indexes normally faster than non-clustered indexes

For instance, in MySQL InnoDB, the clustered index is our primary key. If you don’t define a primary key, InnoDB will try to use a unique non-nullable index instead.

Pay Attention!!!

• In MySQL InnoDB, choosing the right PK is very important

• Should not insert PK values randomly => need to reorder data, increase Disk I/O, disk fragmentation

VI. Covering Index

Indexes need to be designed for the whole query, not just the WHERE clause.

A Covering Index is:

• An index that covers all data needed for a query (only B-tree can be used to cover indexes)

• An index reduces Disk I/O and latency because no need to look up into disk.

So, please query only what you need, should not query redundant data (SELECT * FROM).

E.g.: In MySQL InnoDB, when you issue a query that is covered by an index (an index-covered query), you’ll see “Using index” in the Extra column in EXPLAIN:

VII. Redundant and Duplicate Index

You should analyze your database indexes and remove duplicated and unused indexes. It helps our database to not need to maintain redundant indexes and reduce the impact for INSERT, UPDATE, and DELETE operations.

E.g.: In MySQL InnoDB, The best way to identify unused indexes is with performance_schema and sys:

Notice that if there is an index on (A, B), another index on (A) would be redundant because it is a prefix of the first index.

In most cases, you don’t want redundant indexes, and to avoid them you should extend existing indexes rather than add new ones. However, there are times when you’ll need redundant indexes for performance reasons. Extending an existing index might make it much larger and reduce performance for some queries.

VIII. Partial Index

So far we have only discussed which columns to add to an index. With partial (PostgreSQL) or filtered (SQL Server) indexes you can also specify the rows that are indexed.

A partial index is useful for commonly used where conditions that use constant values - like the status code in the following example:

Queries like this are very common in queuing systems to fetch all unprocessed tasks.

Benefits

• Specify the rows that are indexed

• Reduce disk space and index size

Pay Attention!!!

• Only Oracle, MongoDB, and PostgreSQL support

• Very common in queuing systems

IX. Update Index Statistic

Over time, our data and indexes can become fragmented, which might reduce performance. So, we can consider running:

• ANALYZE TABLE: calculates statistics for indexes

  In MySQL, ANALYZE TABLE returns a result set with the columns shown in the following table.

E.g.: ANALYZE TABLE users

• OPTIMIZE TABLE: reorganizes the physical storage of table data and associated index data, to reduce storage space and improve I/O efficiency when accessing the table.

  In MySQL, OPTIMIZE TABLE returns a result set with the columns shown in the following table.

  

  For InnoDB tables, OPTIMIZE TABLE is mapped to ALTER TABLE ... FORCE, which rebuilds the table to update index statistics and free unused space in the clustered index.

  

  Pay Attention!!!

  • Think twice before acting, especially on your Prod databases.

  • Need to be reviewed by database experts because it can cause downtime.

X. No Index

Last but not least, indexing is not a silver bullet. Sometimes, you get stuck improving your index performance. Please stay calm and think about alternative solutions rather than trying hard with indexing, such as:

• Give up :D

• Use suitable databases for your requirements

  E.g.: To support aggregate queries, please use OLAP databases instead of OLTPs.

• Build temp tables for heavy queries

• Partition your tables, and archive data frequently (based on data retention)

• Combine multiple databases

• Apply CDC (Change Data Capture) to replicate data and scale your read performance.

XI. Conclusion

We just covered 10 strategies for effective indexing and achieving high query performance. I hope this article is useful for anyone working with indexes.

Bear in mind that each database and its storage engines have different index implementations and support different index types. Therefore, be mindful of the database type, storage engine, and version you are using before creating/optimizing your indexes.

XII. References

• High Performance MySQL 4th edition

• SQL Performance Explained

Most of the hard issues come from databases when your system grows.

The bigger your system, the harder your database issues. Index, schema design, partition, and replication,... are important techniques to work with databases.

In particular, mastering indexing is a MUST for backend engineers.

I. Introduction

Indexes are data structures that storage engines use to find rows quickly. This is the most powerful way to improve query performance.

• Store in a separate space with source data

• Index Creation does not change the source data

  Databases create a copy of the column that we create index and link to the original data

• Index is implemented at storage engine layer

  Each database, storage engine supports different index types

A backend engineer needs to master database indexing, or someday they headaches with database issues when their system grows x5 x10.

Benefits

• Improve the search performance of SELECT

• Reduce server’s workload and I/O of disk

Notes: Index also works for UPDATE queries with WHERE when using indexes to look up the record and update

E.g. UPDATE … WHERE id = 1

Limitation

• More space for index (disk, memory)

• INSERT, UPDATE, DELETE queries are slower due to index maintenance

• Too many indexes cause a slowdown in the database server

• Confuses when the optimizer selects an execution strategy

II. Data Structure of Index

There are a lot of index types. However, in terms of structure, we have 2 most common: B-tree and Hash. So I will focus on these two types in this post.

1. B-tree index

This is a Balanced Tree (not a binary tree). Each node contains N values and pointers to the below nodes. Leaf nodes’ values are sorted and each leaf node contains a pointer to the original table.

However, in modern databases such as MySQL and PostgreSQL, when we talk about B-tree, it actually is a B+ tree.

In B+ tree, leaf nodes are connected by a DOUBLE LINKED LIST. So it works more efficiently than B-tree in the case of range queries.

Each leaf node is stored in a block (page) – the smallest unit of the database. Each storage engine has a different block size.

E.g. MySQL (InnoDB) has a default block size of index = 16kB.

a. Searching on B+-tree

The below picture shows an index fragment to illustrate a search for the key "57". The tree traversal starts at the root node on the left-hand side. Each entry is processed in ascending order until a value is greater than or equal to (>=) the search term (57). Then the database follows the reference to the corresponding branch node and repeats the procedure until the tree traversal reaches a leaf node.

From leaf nodes, we have connections to the table data. Unlike the index, the table data is not sorted at all.

b. Supported Query Types

As I said, each database or storage engine supports different index types. So within the scope of this post, let's assume that we are working with MySQL + InnoDB engine.

we have a composite index of 2 columns employee_id and subsidiary_id.

• Match full value

  Your database will use your index if you query by full values (=)

  E.g.

  • WHERE employee_id = 1

  • WHERE employee_id = 123 and subsidiary_id = 20

• Match leftmost prefix

  Your database can use a composite index when searching with the leading (leftmost) columns. An index with three columns can be used when searching for the first column, when searching with the first two columns together, and when searching using all columns.

  E.g.

  • OK: WHERE employee_id = 1

  • NOT OK: WHERE subsidiary_id = 20

• Match range condition

  Your database will use your index if you query by a range condition on the first column.

  E.g.

  • OK: WHERE employee_id >= 123 and employee_id < 125

  • NOT OK: WHERE subsidiary_id > 20

• Match equal in the left and range in the right column

  E.g.

  • OK : WHERE employee_id = 123 and subsidiary_id > 20

  • NOT OK : WHERE employee_id > 123 and subsidiary_id = 20

    \==> Only use index for employee_id search.

• LIKE Operator (leftmost)

  LIKE filters can only use the characters before the first wildcard during tree traversal. The remaining characters are just filter predicates that do not narrow the scanned index range.

  The more selective the prefix before the first wildcard is, the smaller the scanned index range becomes.

  

  E.g.

  • OK: WHERE name LIKE ‘WI%ND’

  • NOT OK: WHERE name LIKE ‘%WIND%’

• Covering Indexes

  The index covers the entire query so it is also called a covering index. It prevents table access and runs pretty fast.

  E.g. SELECT subsidiary_id FROM Employee WHERE employee_id > 123

• Index Merge

  There are queries where a single index cannot do a perfect job.

  E.g. queries with two or more independent range conditions

  

  You can of course accept the filter predicate and use a multi-column index nevertheless. That is the best solution in many cases anyway.

  But what I want to say here is we have another option: two separate indexes, one for each column. Then the database must scan both indexes first and then combine the results.

• Sorting/Group By

  Your database will use your index in case of ordering or grouping because index data is sorted.

  E.g. Select id, name FROM users ORDER BY id DESC limit 10

• Function Indexes

  You must use the same function of your query to create the index.

  E.g.

  Your query is:

  SELECT id, fullname FROM users WHERE UPPER(fullname) = UPPER(‘Pham Duy Hieu’)

  \=> You need to create an index with the UPPER function:

  CREATE INDEX idx_name on users (UPPER(fullname))

c. Index with NULL values

When we create the index for Nullable columns, MySQL needs 1 byte to detect null/non-null values. Besides, optimizing queries (internal) in MySQL can be more challenging.

E.g. I have a users table

You can check the key length in Explain result:

explain select * from users where departure_id = 1;

You can see that departure_id is interger 4 bytes, right? However, the real key length is 5 bytes due to 1 byte for its length.

So you should set NOT NULL for your columns whenever you can. Even when you do need to store a “no value” in a table, you might not need to use NULL. Perhaps you can use zero, a special value, or an empty string instead.

However, don’t be too afraid of using NULL when you need to represent an unknown value. In some cases, it’s better to use NULL than a magical constant.

2. Hash Index

The index type is based on the Hashing technique to build a Hash Table and store indexed data. However, it is only supported by MySQL (Memory), Oracle and PostgreSQL.

If you are using MySQL InnoDB and you create a HASH index, InnoDB silently changes HASH to Btree.

Here are some points of a hash index:

• Good performance with "="

• Don't support range queries (>, <, >=, <=) and sorting, group by

• Hash Collision: should not be used for columns with many duplicated values

• More space for hashing values (than B-tree)

We have just walked through the most common data structures for indexing and the supported query types accordingly. In the next post, I will share common indexing strategies allowing you to start practicing on your own. See youuuuu!

(continued)

IV. References

• High Performance MySQL, 4th Edition

• SQL Performance Explained Everything Developers Need to Know about SQL Performance

I. Pros and Cons

You need to analyze the pros and cons of our migration. In my experience, there are some points you should review:

1. Pros

Java 17 Baseline

You’ll need to upgrade to JDK 17 before you can develop Spring Boot 3.0 applications. This means you can take advantage of the latest features and performance improvements that Java 17 offers.

GraalVM Native Image Support

GraalVM Native Images provide a new way to deploy and run Java applications. It provides various advantages, like an instant startup and reduced memory consumption (pain points of Spring Boot apps).

Improved observability with Micrometer and Micrometer Tracing

You can check more details here.

2. Cons

Time/Resouces constraints

Migrating to a new major release takes time and resources, especially for testing. This migration affects all your flows so needs to be tested carefully. While you can update your code within a few days, please plan for testing to span more than a week (the duration depends on the size of your project).

Risk of new bugs

As mentioned earlier, the migration affects all your flows. Therefore, if your test coverage doesn't cover all your code, please be careful. Test your end-to-end flows and scrutinize the logs for any new exceptions or discrepancies compared to the previous state.

II. Before we start

If you’re currently running with an earlier version of Spring Boot, I recommend that you upgrade to Spring Boot 2.7 before migrating to Spring Boot 3.0. It minimizes compatibility issues as much as possible.

Review Dependencies

You can review your dependencies and dependency management for 3.x to assess how your project is affected.

For dependencies that are not managed by Spring Boot, you can identify the compatible version before upgrading.

Review Deprecations

Classes, methods and properties that were deprecated in Spring Boot 2.x have been removed in this release. Prior to upgrading, please ensure that you are not calling any deprecated methods.

III. Migrate to Spring Boot 3 and Java 17

1. Spring Boot Template Project

I will use my project as an example for you guys to share how I migrated from Spring Boot 2.7 (Java 11) to Spring Boot 3 and Java 17.

Github: https://github.com/hieubz/spring-boot-template-project

This project includes the implementation of common backend features, designed to assist both myself and other Spring Boot developers in coding more efficiently. For further details, you can read more here.

2 . Migration Steps

Configuration Properties Migration

Let's add the migrator by adding the following to your Maven pom.xml:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-properties-migrator</artifactId>
    <scope>runtime</scope>
</dependency>

This will analyze your application’s environment and print diagnostics at startup console logs. Then you can based on that update your properties accordingly.

Update Dependencies

• We start with the parent pom spring-boot-starter-parent and Java version

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>3.1.5</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>

    <properties>
        <java.version>17</java.version>
    </properties>

Tips: We shouldn't specify the version of Spring Data JPA, Spring Web, Spring Data Redis,... because their compatible versions are already declared in the parent POM.

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-jpa</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

• If you are working with MySQL, let's replace your mysql-connector-java by:

        <dependency>
            <groupId>com.mysql</groupId>
            <artifactId>mysql-connector-j</artifactId>
            <version>8.1.0</version>
        </dependency>

• If you are using logback for logging, let's update it to v1.4.11

        <dependency>
            <groupId>ch.qos.logback</groupId>
            <artifactId>logback-classic</artifactId>
            <version>1.4.11</version>
        </dependency>

• If you are using Openfeign for integrations, let's update it to v4.x

        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-openfeign</artifactId>
            <version>4.0.4</version>
        </dependency>

• If you are using Spring JDBC, let's update it to v6.x

        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-jdbc</artifactId>
            <version>6.0.12</version>
        </dependency>

• If you are using Jackson for data binding, let's update it to v2.15.x

        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-databind</artifactId>
            <version>2.15.3</version>
        </dependency>

        <!--  support Java 8 Date/time Serialize/Deserialize  -->
        <dependency>
            <groupId>com.fasterxml.jackson.datatype</groupId>
            <artifactId>jackson-datatype-jsr310</artifactId>
            <version>2.15.3</version>
        </dependency>

• If you are using Redisson as a Redis client, let's update it to v3.24.x

        <dependency>
            <groupId>org.redisson</groupId>
            <artifactId>redisson-spring-boot-starter</artifactId>
            <version>3.24.3</version>
        </dependency>
        <dependency>
            <groupId>org.redisson</groupId>
            <artifactId>redisson-spring-data-31</artifactId>
            <version>3.24.3</version>
        </dependency>

• You should use the default spring-boot-starter-test for unit testing. However, If you are customizing mockito-core, let's update it to v5.3.x

        <dependency>
            <groupId>org.mockito</groupId>
            <artifactId>mockito-core</artifactId>
            <version>5.3.1</version>
            <scope>test</scope>
        </dependency>

• If you are using jjwt for authentication, let's update it to 0.12.x

        <dependency>
            <groupId>io.jsonwebtoken</groupId>
            <artifactId>jjwt</artifactId>
            <version>0.12.3</version>
        </dependency>

• If you are using Springdoc for API documentation, let's replace your springdoc-openapi-ui by:

        <dependency>
            <groupId>org.springdoc</groupId>
            <artifactId>springdoc-openapi-starter-webmvc-ui</artifactId>
            <version>2.2.0</version>
        </dependency>

• If you are using spring-kafka as your Kafka client, let's update it to v3.0.x

        <dependency>
            <groupId>org.springframework.kafka</groupId>
            <artifactId>spring-kafka</artifactId>
            <version>3.0.10</version>
        </dependency>

Rebuild and change the code

• After updating your dependencies, let's rebuild your code first and check for any issues

    mvn clean package
  

• Spring Boot 3.0 has migrated from Java EE to Jakarta EE APIs for all dependencies. So you should face the javax issue in your first build:

  

  Then you just need to replace all javax in your imports by jakarta (should use Ctrl+Shift+R to replace on IntelliJ)

• 

  If the Spring migrator is working, you can see a WARNING log in the startup console:

  

  Simply update as suggested.

• In Hibernate 6, you can use MySQLDialect for all MySQL versions (MySQL5Dialect, MySQL8Dialect have been deprecated)

spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.MySQLDialect

• JPA SpringPhysicalNamingStrategy is replaced by CamelCaseToUnderscoresNamingStrategy

org.springframework.boot.orm.jpa.hibernate.SpringPhysicalNamingStrategy

==> org.hibernate.boot.model.naming.CamelCaseToUnderscoresNamingStrategy

• Spring Security is not working if you are using WebSecurityConfigurerAdapter (deprecated). Besides, in v3.x, you have to use lambda to configure filterChain.

So you have to remove the extension of WebSecurityConfigurerAdapter class.

// Now
@Configuration
@EnableWebSecurity
@EnableGlobalMethodSecurity(prePostEnabled = true)
public class WebSecurityConfig extends WebSecurityConfigurerAdapter {}

// Then without extension
@Configuration
@EnableWebSecurity
@EnableMethodSecurity
public class WebSecurityConfig {}

Then do several updates for AuthenticationManager bean creation:

// Now with the class extends WebSecurityConfigurerAdapter
  @Bean
  @Override
  public AuthenticationManager authenticationManagerBean() throws Exception {
    // Get AuthenticationManager bean
    return super.authenticationManagerBean();
  }

// Then without extension
  @Bean
  public AuthenticationManager authenticationManager(AuthenticationConfiguration authConfig) throws Exception {
    return authConfig.getAuthenticationManager();
  }

And SecurityFilterChain:

/** We have to use Lambda for SecurityFilterChain configuration **/

// Now with the class extends WebSecurityConfigurerAdapter
  @Override
  protected void configure(HttpSecurity http) throws Exception {
      http.cors().and().csrf().disable();
      http.authorizeRequests().antMatchers(AUTH_WHITELIST).permitAll().anyRequest().authenticated();
      http.sessionManagement().sessionCreationPolicy(SessionCreationPolicy.STATELESS);
      http.exceptionHandling().authenticationEntryPoint(authEntryPointJwt);
      http.addFilterBefore(jwtAuthenticationFilter(), UsernamePasswordAuthenticationFilter.class);
  }

// Then with lambda
  @Bean
  protected SecurityFilterChain configure(HttpSecurity http) throws Exception {
      http.cors(AbstractHttpConfigurer::disable).csrf(AbstractHttpConfigurer::disable);
      http.authorizeHttpRequests(auth -> auth.requestMatchers(AUTH_WHITELIST).permitAll().anyRequest().authenticated());
      http.sessionManagement(s -> s.sessionCreationPolicy(SessionCreationPolicy.STATELESS));
      http.exceptionHandling(ex -> ex.authenticationEntryPoint(authEntryPointJwt));
      http.addFilterBefore(jwtAuthenticationFilter(), UsernamePasswordAuthenticationFilter.class);
      return http.build();
  }

• Powermock is not working with Spring Boot 3 and JDK 17. Its latest update is in Feb 2022. So you need to move to Mockito if you are using Powermock.

• The jjwt library updates its API, so we update our code:

// Now
  public String generateJwtToken(Long userId) {
    return Jwts.builder()
        .setSubject(userId.toString())
        .setIssuedAt(new Date())
        .setExpiration(new Date(new Date().getTime() + jwtExpirationMs))
        .signWith(SignatureAlgorithm.HS256, jwtSecret.getBytes(StandardCharsets.UTF_8))
        .compact();
  }


// Then
  public String generateJwtToken(Long userId) {
    return Jwts.builder()
        .subject(userId.toString())
        .issuedAt(new Date())
        .expiration(new Date(new Date().getTime() + jwtExpirationMs))
        .signWith(Keys.hmacShaKeyFor(jwtSecret.getBytes(StandardCharsets.UTF_8)), Jwts.SIG.HS256)
        .compact();
  }

And the JWT parser:

// Now
public Claims getJwtTokenClaim(String jwt) {
   return Jwts.parser()
        .setSigningKey(jwtSecret.getBytes(StandardCharsets.UTF_8))
        .parseClaimsJws(jwt)
        .getBody();
}

// Then
public Claims getJwtTokenClaim(String jwt) {
    return Jwts.parser()
        .verifyWith(Keys.hmacShaKeyFor(jwtSecret.getBytes(StandardCharsets.UTF_8)))
        .build()
        .parseSignedClaims(jwt)
        .getPayload();
}

• Finally, rebuild and check for any issues. In my project, there are no issues left, so I stop updating the code here.

Testing

Since this is a major upgrade, carefully test all APIs for discrepancies or exceptions. Review them one by one and monitor logs.

IV. Conclusion

We just completed the migration to Spring Boot 3 and JDK 17. There are many issues during this process, so stay calm and get things done :))). I hope this is helpful for anyone planning the migration.

Hieu Pham.

Nhưng có một ngư dân, mỗi lần anh chở cá về chúng đều còn sống. Vì thế cá của anh bán được giá cao gấp đôi người khác. Mấy năm sau, người ngư dân này đã trở thành một phú ông giàu có vang danh gần xa. Đến khi bệnh nặng không thể ra biển được nữa, người ngư dân mới đem bí mật của mình nói lại với con trai về bí quyết giúp cá chình không chết:

Trong khoang thuyền chứa đầy cá chình, ông đã bỏ một con cá nheo vào đó. Trong tự nhiên, cá chình luôn đánh nhau với nheo.

Để chống lại những đợt công kích của cá nheo, cá chình buộc phải cố gắng nghênh chiến. Trong tình trạng đấu tranh như vậy, bản năng sống của cá chình sẽ được huy động tối đa, cho nên nó vẫn còn sống khi vào đến bờ.

Người ngư dân còn nói với con trai, nguyên nhân khiến cá chình chết là vì chúng biết chúng đã bị bắt, trước mắt chúng chỉ có cái chết, hy vọng sống đã bị dập tắt, cho nên ở trong khoang không được bao lâu thì chúng đều chết hết.

Cuối cùng bác ngư dân khuyên các con, phải dũng cảm đấu tranh, chỉ có đấu tranh, cuộc sống mới tràn đầy niềm tin và hy vọng.

Ở Nhật Bản, khi trẻ con vừa mới hiểu chuyện, câu chuyện đầu tiên mà cha mẹ kể cho chúng chính là câu chuyện về cá chình.

Tất cả những đứa trẻ Nhật Bản từ nhỏ đã được truyền niềm tin: Chỉ có dũng cảm chiến đấu, mới có được thành công và hy vọng!

(Sưu tầm)

This article has 4 parts:

• General programming: how to use variables, control structures, libraries, data types,... effectively.

• Lambdas and Streams: how to make the best use of functional interfaces, lambdas, and method references.

• Exceptions: guidelines for using exceptions effectively.

• Concurrency: write clear and correct concurrent programs.

Ok, let's go!!!

I. General Programming

a. Prefer for-each loops to traditional for loops

for-each loops get rid of the clutter and the opportunity for error by hiding the iterator or index variable. There is no performance penalty for using for-each

However, there are some common situations where you can’t use for-each:

• need the array index in order to do something

• need to traverse multiple collections in parallel, then you need explicit control over the iterator or index variable

b. Prefer primitive types to boxed primitives

• Primitives are more time and space efficient than boxed primitives

• Applying the == operator to boxed primitives is almost always wrong => be careful when comparing boxed primitives.

• When you mix primitives and boxed primitives in an operation, the boxed primitive is auto-unboxed. If a null object reference is auto-unboxed, you get a NullPointerException

• Repeatedly boxed and unboxed causing performance degradation.

c. Avoid Strings where other types are more appropriate

• If it’s numeric, it should be translated into the appropriate numeric type, such as int, long, float, or BigInteger

• If it’s the answer to a yes-or-no question, it should be translated into an appropriate enum type or a boolean

d. Beware the performance of string concatenation

Strings are immutable, so when two strings are concatenated, the contents of both are copied and Java creates a new String

To achieve acceptable performance, use a StringBuilder in place of a String

e. Refer to objects by their interfaces

If appropriate interface types exist, then parameters, return values, variables, and fields should all be declared using interface types

Your program will be much more flexible to switch implementations. However, it is entirely appropriate to refer to an object by a class rather than an interface if no appropriate interface exists. If there is no appropriate interface, just use the least specific class in the class hierarchy that provides the required functionality**.**

II. Lambdas and Streams

a. Prefer lambdas to anonymous classes

In Java 8, the language formalized the notion that interfaces with a single abstract method are special and deserve special treatment. These interfaces are now known as functional interfaces, and the language allows you to create instances of these interfaces using lambda expressions, or lambdas for short.

However, lambdas lack names and documentation. If a computation isn’t self-explanatory or exceeds a few lines, don’t put it in a lambda. One line is ideal for lambda, and three lines are a reasonable maximum.

b. Prefer method references to lambdas

Java provides a way to generate function objects even more succinct than lambdas: method references

should become:

However, where method references are shorter and clearer, use them; where they aren’t, stick with lambdas.

c. Favor the use of standard functional interfaces

If one of the standard functional interfaces does the job, you should generally use it in preference to a purpose-built functional interface.

Of course you need to write your own if none of the standard ones does what you need, for example if you require a predicate that takes three parameters.

d. Use streams judiciously

• Overusing streams makes programs hard to read and maintain.

• In the absence of explicit types, careful naming of lambda parameters is essential to the readability of stream pipelines.

• Using helper methods is important for readability in stream pipelines.

f. Prefer Collection to Stream as a return type

When writing a method that returns a sequence of elements, remember that some of your users may want to process them as a stream while others may want to iterate over them

\=> Collection or an appropriate subtype is generally the best return type

g. Use caution when making streams parallel

Java 8 introduced streams, which can be parallelized with a single call to the parallel method.

• Parallelizing a pipeline is unlikely to increase its performance if the source is from Stream.iterate, or the intermediate operation limit is used

• Performance gains from parallelism are best on streams over ArrayList, HashMap, HashSet, and ConcurrentHashMap instances; arrays; int ranges; and long ranges. What these data structures have in common is that they can all be accurately and cheaply split into subranges of any desired sizes, which makes it easy to divide work among parallel threads

• Do not even attempt to parallelize a stream pipeline unless you have good reason to believe that it will preserve the correctness of the computation and increase its speed

III. Exceptions

a. Use checked exceptions for recoverable conditions and runtime exceptions for programming errors

• Use checked exceptions for conditions from which the caller can reasonably be expected to recover.

  Ex: For example, suppose a checked exception is thrown when an attempt to make a purchase with a gift card fails due to insufficient funds. The exception should provide an accessor method to query the amount of the shortfall. This will enable the caller to relay the amount to the shopper.

• If a program throws an unchecked exception or an error, it is generally the case that recovery is impossible and continued execution would do more harm than good.

  Ex: ArrayIndexOutOfBoundsException, NullPointerException

• If it isn’t clear whether recovery is possible, you’re probably better off using an unchecked exception

b. Avoid unnecessary use of checked exceptions

• Overuse of checked exceptions places a burden on the user of the API

• If callers won’t be able to recover from failures, throw unchecked exceptions. If recovery may be possible and you want to force callers to handle exceptional conditions, first consider returning an optional.

However, if optional provides insufficient information in the case of failure, you should throw a checked exception.

c. Favor the use of standard exceptions

The Java libraries provide a set of exceptions that cover most of the exception-throwing needs of most APIs.

This table summarizes the most commonly reused exceptions:

Do not reuse Exception, RuntimeException, Throwable, or Error directly.

d. Throw exceptions appropriate to the abstraction

Higher layers should catch lower-level exceptions and, in their place, throw exceptions that can be explained in terms of the higher-level abstraction (exception translation)

e. Include failure-capture information in detail messages

• To capture a failure, the detail message of an exception should contain the values of all parameters and fields that contributed to the exception.

Ex: The detail message of an IndexOutOfBoundsException should contain the lower bound*, the* upper bound*, and the* index value that failed to lie between the bounds

• Do not include passwords, encryption keys,... in detail messages

f. Strive for failure atomicity

A failed method invocation should leave the object in the state that it was in prior to the invocation. There are several ways to achieve this effect:

• Design immutable objects

• For methods that operate on mutable objects ==> check parameters for validity before performing the operation (exceptions will be thrown before object modification commences)

  

g. Don't ignore exceptions

When the designers of an API declare a method to throw an exception, they are trying to tell you something. Don’t ignore it!

If you choose to ignore an exception, the catch block should contain a comment explaining why it is appropriate to do so, and the variable should be named ignored

IV. Concurrency

a. Synchronize access to shared mutable data

The synchronized keyword ensures that only a single thread can execute a method or block at one time.

• when multiple threads share mutable data, each thread that reads or writes the data must perform synchronization

  

  without synchronization, there is no guarantee that one thread’s changes will be visible to another thread

• volatile modifier performs no mutual exclusion, but guarantees that any thread that reads the field will see the most recently written value

  

However, you have to be careful when using volatile. For example, in the following method, the problem is that the increment operator (++) is not atomic. It performs two operations (volatile still works if performs one operation) on the nextSerialNumber field: first it reads the value, and then it writes back a new value:

If a second thread reads the field between the time a thread reads the old value and writes back a new one, the second thread will see the same value as the first and return the same serial number ==> computes the wrong results.

One way to fix it is to use AtomicLong.

The best way to avoid the problems discussed is not to share mutable data.

b. Avoid excessive synchronization

• Excessive synchronization can cause reduced performance, deadlock,...

• Inside a synchronized region, do not invoke a method that is designed to be overridden, or one provided by a client in the form of a function object

  \=> has no knowledge of what the method does and has no control over it.

There are some ways to move the alien method invocations out of the synchronized block:

• taking a “snapshot” of shared mutable data => safely traversed without a lock

• use concurrent collections*: CopyOnWriteArrayList*, ConcurrentHashMap,...

As a rule, you should do as little work as possible inside synchronized regions! If you must perform some time-consuming activity, find a way to move it out of the synchronized region.

If you are writing a mutable class, you have two options: you can omit all synchronization and allow the client to synchronize externally if concurrent use is desired, or you can synchronize internally, making the class thread-safe

c. Prefer executors, tasks, and streams to threads

Instead of creating and managing threads manually, you can create a thread pool with a fixed or variable number of threads. I prefer ThreadPoolExecutor or ThreadPoolTaskExecutor (in SpringBoot) for a heavily loaded production server.

d. Prefer concurrency utilities to wait and notify

Instead of using wait and notify, you should use the higher-level concurrency utilities in java.util.concurrent package: Executor, concurrent collections and synchronizers.

• use ConcurrentHashMap in preference to Collections.synchronizedMap

• For interval timing, use System.nanoTime rather than System.currentTimeMillis

  (more accurate and more precise and is unaffected by adjustments to the system’s real-time clock)

If you have to maintain legacy code that uses wait and notify, always use the wait loop idiom (inside a synchronized region) to invoke the wait method; never invoke it outside of a loop:

e. Use lazy initialization judiciously

The best advice for lazy initialization is “don’t do it unless you need to”. It decreases the cost of initializing a class or creating an instance, at the expense of increasing the cost of accessing the lazily initialized field.

Under most circumstances, normal initialization is preferable to lazy initialization.

If you must initialize a field lazily in order to achieve your performance goals or break a harmful initialization circularity:

• For instance fields, use double-check idiom (singleton pattern)

  

• For static fields, use lazy initialization holder class idiom

  

f. Don't depend on the thread scheduler

When many threads are runnable, the thread scheduler determines which ones get to run and for how long. Any reasonable operating system will try to make this determination fairly, but the policy can vary. Therefore, well-written programs shouldn’t depend on the details of this policy.

The best way to write a robust, responsive, portable program is to ensure that the average number of runnable threads is not significantly greater than the number of processors. This leaves the thread scheduler with little choice: it simply runs the runnable threads till they’re no longer runnable. Note that the number of runnable threads isn’t the same as the total number of threads, which can be much higher. Threads that are waiting are not runnable.

In terms of the Executor Framework, this means sizing thread pools appropriately and keeping tasks short, but not too short, or dispatching overhead will harm performance.

Phewww!!! I've just shared with you guys some important tips and tricks to work with Java efficiently. If you find it helpful, please share it with everyone to join hands in building a stronger tech community in Vietnam.

References

1. Effective Java 3rd Edition

However, after several months, I might forget how to implement that feature. Consequently, I end up having to revisit internet resources, conduct research once again, and reattempt the implementation.

The above situation is very common, not just for me but also among my friends and colleagues. We waste a lot of time recalling and researching solutions - things could be synthesized in a template project for the next implementation.

So I have developed this project which includes implementations designed to assist both myself and other Spring Boot developers in seamlessly diving into coding. Each commit within the project represents the implementation of a specific technique.

Link: https://github.com/hieubz/spring-boot-template-project

Tech stack: Java 11, Spring Boot 2.7.x (upgraded to Java 17, Spring Boot 3.1.x)

Every time I need to implement a feature in this list, I can search by feature keywords (such as Redis, retry, email,...) to find the corresponding commit, then apply the code of that commit for my feature. No need to search and remember boilerplate code anymore!

We will have basic features here:

or advanced things like:

There are also modules related to Unit Test, Monitoring:

I hope this project helps Spring Boot developers increase their productivity, allowing them to allocate more time to other important activities.

I plan to upgrade to Spring Boot 3 and Java 17 in the near future.

See yaaa!

Cuốn sách được chia làm 5 phần: Những thế lưỡng nan, thử nghiệm, đánh giá, học tập, thực thi. Năm phần cũng chính là 5 bước trong mô hình vòng lặp DevUP mà tác giả đề xuất. Nói nôm na nó là việc bạn lựa chọn một việc gì đó, thử nghiệm và thực thi rồi rút ra bài học, sau đó cứ tiếp tục vòng lặp đó hàng ngày để ngày một tiến bộ.

Experiment - Valuation - Unlearn - Performance Vòng lặp 4 bước này sẽ giúp bạn liên tục phải vận động, học hỏi cái mới, rèn luyện kỹ năng, mở rộng các mối quan hệ với cộng đồng trong và ngoài ngành IT. Mỗi lần đi hết một cung tròn, chính là một lần LTV nâng cấp được trình độ, kỹ năng của mình. Nó có thể tốn 1 tháng, cũng có thể chỉ 1 tuần.

Bạn cũng sẽ biết cách đánh giá ROI - Return Of Investment mỗi khi đưa ra quyết định.

VD: bạn đã có nền tảng Java 8 rồi thì giữa việc học thêm Java 11 và học mới Golang, cái nào có ROI cao hơn? Cái đó tùy thuộc vào hoàn cảnh của bạn. Nếu công ty bạn chuộng Java và sắp tới có xu hướng sử dụng Java 11, thì đương nhiên việc học Java 11 có ROI cao hơn. Nhưng nếu bạn có mong muốn chuyển sang môi trường mới và Go là ngôn ngữ được chuộng ở các công ty đó, thì đầu tư vào Go ở thời điểm hiện tại là một lựa chọn đúng đắn. Còn nếu bạn thấy Return từ 2 việc trên là như nhau, bạn nên học Java 11 vì đơn giản Investment cho việc học Java 11 ít hơn, do đó ROI tốt hơn.

Và một điều nữa là một chuyên môn giỏi phải đi kèm với khả năng nhìn toàn cảnh thì mới mang lại kết quả. Nếu không, có thể một phần mềm hoàn hảo được sinh ra nhưng không có người dùng.

Trên đây là một vài nhận xét về cuốn sách. Giờ là tới chi tiết từng phần mà mình tóm tắt lại theo ý hiểu của mình, sau còn đọc lại.

I. Hiểu những thế lưỡng nan

Cuộc sống luôn đi kèm với những hoàn cảnh éo le, không rõ trắng đen, đòi hỏi bạn phải ra quyết định. Đúng hay sai à? còn tùy vào góc nhìn.

Bản thân mình cũng từng đứng trước nhiều ngã ba đường: đi làm Dev hay BA, đi hát mỗi tối kiếm tiền, vui nhưng bấp bênh hay theo ngành IT đã chọn.

Đấy là đối với định hướng nghề nghiệp. Còn chỉ riêng trong ngành IT cũng có hàng chục thế lưỡng nan được tác giả đưa ra đánh giá.

Chung quy lại, đứng trước những lựa chọn, tác giả khuyên các LTV nên cân nhắc được-mất trong dài hạn thay vì nhìn vào những thứ trước mắt như lương bổng hay vì name của một công ty. Làm việc ở nước ngoài về lâu dài chưa hẳn đã phải là tốt, hay ở lại trong nước cũng không hẳn là tụt hậu. Tác giả đã chỉ ra ưu/nhược điểm của từng lựa chọn, cho chúng ta cái nhìn từng trải của một người lâu năm trong nghề.

Đi hay ở, tập đoàn lớn hay startup, đi sâu hay đi rộng, lương tăng bao nhiêu là đủ,... tùy ở việc bạn muốn gì, muốn đạt được gì trong 5-10 năm tới. Mục tiêu dài hạn sẽ ảnh hưởng tới những quyết định của bạn trong hiện tại.

An "easy" short-term decision may introduce long-term problems

- Noam Wasserman -

II. Thử Nghiệm

Khi đã ở trong thế lưỡng nan, LTV chỉ còn cách lựa chọn theo những gì linh tính mách bảo (cái này là tôi thêm vào :D). Vì nếu cứ phân vân thì cũng vấn đề cứ nằm đó. Chi bằng ta cứ chọn và thử nghiệm.

Hãy thử nghiệm một thiết kế, công cụ, hay công nghệ trong những pet project của riêng mình. Khi tầm ảnh hưởng của bạn trong tổ chức tăng lên, hãy mạnh dạn đề xuất những kết quả từ những thử nghiệm cá nhân.

Theo kinh nghiệm của tôi, bạn sẽ không thể trải nghiệm đủ những công nghệ, dự án, domain mà bạn muốn nếu chỉ làm những gì được giao. Hãy mạnh dạn xin chuyển team nếu thực sự muốn trải nghiệm ở vị trí khác. Và chăm chỉ thử nghiệm trong những pet project của bạn.

III. Đánh giá

Sau khi đã thử nghiệm thì cũng phải đánh giá kết quả chứ nhỉ: đúng hay sai, mình đang đứng ở đâu, học được điều gì? Và sau những thử nghiệm và đánh giá đó, con đường của bạn ngày càng rõ ràng, hoặc chí ít bạn cũng đã có thêm dữ kiện cho những quyết định sắp tới.

Đánh giá xem mình đang Junior hay Senior, lựa chọn nào có ROI tốt hơn, góc nhìn của mình đã toàn cảnh chưa, phản hồi từ đồng nghiệp hay cấp trên như thế nào?

IV. Học tập

Sau khi đã biết mình đang ở đâu, còn thiếu gì, thì là lúc bạn cần bổ sung kiến thức, kỹ năng còn thiếu hoặc chưa hoàn chỉnh. Học tập cũng có nghĩa là sắp đặt, cấu trúc lại hệ thống kiến thức đã có nhằm tiếp thu những kiến thức mới. Nôm na là unlearn, xem lại những gì mình đã học và từ bỏ nếu chúng không còn đúng đắn.

Tác giả đã đưa ra ý kiến về những điều tưởng chừng như là chân lý, thường như cơm bữa đối với LTV. Từ chuyện tỏ ra lập dị, hay trễ hẹn, tâm lý kiểu gì cũng có bug, tư tưởng đòi hỏi yêu cầu hay quy trình phải rõ ràng,... cho tới tư tưởng LTV chỉ làm tới 40 tuổi. Những lối suy nghĩ này tưởng như là bình thường nhưng lại trở nên bất thường hoặc sai trong thời điểm hiện tại.

Ngay bản thân mình cũng đã từng nghĩ như vậy khi còn đi học. Cho tới khi đi làm, gặp nhiều người anh/chị rất giỏi trong nghề, mình mới thấy những điều nhiều LTV hay chẹp miệng cho qua hay coi là hiển nhiên đó thực chất đang khiến họ trở nên xuề xòa, dễ dãi, thiếu chuyên nghiệp, và xa hơn là bị tụt lại phía sau.

Đừng cố bám vào những điều xưa cũ. Hãy học cách thử thách mọi thứ, từ phong cách code, công nghệ, quy trình, thậm chí, đến cách thức xây dựng hệ thống phần mềm.

V. Thực thi

Có làm thì mới có ăn, không làm mà đòi có ăn thì ...

Đấy, học chán r thì phải thực hành, hành động hàng ngày một cách đúng đắn. Đó chính là lúc LTV chuyển hóa những thử nghiệm thành một tư duy, kỹ năng bền vững của bản thân, thậm chí như một thói quen hay phản xạ vô điều kiện.

Và giá trị của một LTV sau cùng sẽ được đo đếm bằng giá trị kết quả mà anh ta mang lại trong công việc, cho tổ chức, cho cộng đồng.

Do vậy, ngoài việc thực thi công việc, LTV còn phải cải thiện những kỹ năng cơ bản nhưng cực kỳ quan trọng khi làm việc trong một tổ chức như problem solving, quản lý công việc, giao tiếp/cộng tác với đồng nghiệp, hay chỉ đơn giản ở việc biết say no.

Cuối cùng, không ai phát triển được chuyên môn nếu tách mình ra khỏi cộng đồng chuyên môn. Tham gia và đóng góp vào cộng đồng chuyên môn nên là ưu tiên quan trọng của LTV. Hãy theo dõi và học hỏi những LTV giàu kinh nghiệm. Hãy chủ động chia sẻ kiến thức, kinh nghiệm của mình với cộng đồng, qua đó giúp ích cho những người khác và bạn cũng sẽ nhận được những phản biện của những LTV khác để hoàn thiện mình.

Đó là đôi dòng những gì mình nhận xét và tóm tắt lại từ cuốn Dev UP. Nhiều kiến thức lắm, không nhớ hết được, nên chắc thỉnh thoảng phải đọc lại. Ae nào chưa đọc thì lên Tiki mà đặt về nhé. Bổ ích lắm :D

Thôi nay thế thôi nhé, tôi đi chạy bộ đã =)))

-- Hanoi 12/11/2022 18h23 --

You could read Part 1 here: High Performance in MySQL - Part 1

Notes: if you have read High Performance in MySQL book, you could realize that my series is only a note I rewrite from this book. My main goal is that I won't forget what I've learned, and can look it up online whenever I need it.

I. Replication

Replication lets you configure one or more servers as replicas of a master server, keeping their data synchronized with the source copy.

How Replication works

At a high level, replication is a simple three-part process:

1. The source records changes to its data in its binary log as “binary log events.”

2. The replica copies the source’s binary log events to its local relay log.

3. The replica replays the events in the relay log, applying the changes to its own data.

Benefits

• Data distribution: useful for maintaining a copy of your data in a geographically distant location, such as a different data center or cloud region

• Scaling read traffic: distribute read queries across several servers, which works very well for read-intensive applications (need to setup LB)

• Backups: a valuable technique for helping with backups. However, a replica is neither a backup nor a substitute for backups.

• Analytics and Reporting: Using a dedicated replica for reporting/analytics (online analytical processing, or OLAP) queries

• High availability and failover: avoid making MySQL a single point of failure in your application

• Testing MySQL upgrades: common practice to set up a replica with an upgraded MySQL version and use it to ensure that your queries work as expected before upgrading every instance.

Replication Problems and Solutions

• Binary Logs Corrupted on the Source: rebuild your replicas

• Non-unique Server IDs: accidentally configure two replicas with the same server ID => be careful when setting up your replicas

• Undefined Server IDs: will not let you start the replica

• Missing Temporary Tables: use row-based replication | name your temporary tables consistently (prefix with temporary_, for example) and use replication rules to skip replicating them entirely

• Not Replicating All Updates: misuse SET SQL_LOG_BIN = 0 or don’t understand the replication filtering rules, your replica might not execute some updates that have taken place on the source

• Replication Lag:

  • Multithreaded replication

  • Use sharding: scale reads with replicas, scale writes with sharding

  • Turn off sync bin log on replicas (when sharding is not a viable option because of effort or design issues)

Keep it simple. Don’t do anything fancy, such as using replication rings or replication filters, unless you really need to.

II. Table Partitioning

Table Partitioning is the way a MySQL database splits its actual data down into separate tables, but still gets treated as a single table by the SQL layer

Partitioning Rules

• must add the partition key into the primary key

Ex: the primary key must include “created” column

CREATE TABLE userslogs (
    username VARCHAR(20) NOT NULL,
    logdata BLOB NOT NULL,
    created DATETIME NOT NULL,
    PRIMARY KEY(username, created)
)
PARTITION BY RANGE( YEAR(created) )(
    PARTITION from_2013_or_less VALUES LESS THAN (2014),
    PARTITION from_2014 VALUES LESS THAN (2015),
    PARTITION from_2015 VALUES LESS THAN (2016),
    PARTITION from_2016_and_up VALUES LESS THAN MAXVALUE

• the table itself becomes a virtual concept. The partitions hold the data and any indexes are built on the data in the partitions.

• MySQL supports horizontal partitioning but not vertical

• Partition types:

  • Range: it is great because you have groups of known IDs in each table, and it helps range queries.

  • Hash: “load balances” the table, and allows you to write to partitions more concurrently. This makes range queries on the partition key a bad idea.

Benefits

• Deletion: quickly delete data that is no longer needed

• Storage: possible to store more data in one table than can be held on a single disk or file system partition

• Performance: Query data faster when only accessing a smaller volume of data (could apply partition pruning)

III. Scaling MySQL

Scaling is the system’s ability to support growing traffic

Read-Bound Workloads

When adding more application nodes to scale the clients serving requests leads to some database issues:

• High CPU: means the server is spending all of its time processing queries. The higher CPU utilization gets, the more latency you will see in queries.

• Heavy disk read IOPS or throughput: indicating that you are going to disk very often or for large numbers of rows read from disk

\==> adding indexes, optimizing queries, and caching data you can cache

Write-Bound Workloads

There are some examples of write-bound workloads:

• Peak e-commerce season and sales are growing, along with the number of orders to track.

• Signups are growing exponentially (Ex: ChatGPT reaches 100 million users 2 months after launch)

All of these are business use cases that lead to exponentially more database writes that you now have to scale.

\==> scale up (add more RAM, CPU and disk) or scale out (functional sharding)

1.Scaling Read

• Use Replica Read Pools

• A very common way to manage these read pools is to use a load balancer (HAProxy, Nginx) to run a virtual IP that acts as an intermediary for all traffic meant to go to the read replicas

2.Scaling Write with Sharding

• Sharding means splitting your data into different, smaller database clusters so that you can execute more writes on more source hosts at the same time.

• Do not split based on the structure of the engineering team. That will always change at some point. Do split tables based on business function

• Most applications shard only the data that needs sharding—typically, the parts of the data set that will grow very large. And not just the data that is growing rapidly but also the data that logically belongs with it and will regularly be queried at the same time (partitioning)

• Do not shy away from tackling spots where separate business concerns have been intermingled in the data and you need to advocate for not just data separation but also application refactoring and introducing API access across those boundaries.

Choosing a Partitioning Scheme

• A good partitioning key is usually the primary key of a very important entity in the database. These keys determine the unit of sharding. For example, if you partition your data by a user ID or a client ID, the unit of sharding is the user or client.

• Diagram your data model with an entity-relationship diagram or an equivalent tool that shows all the entities and their relationships. Try to lay out the diagram so that the related entities are close together.

• Consider your application’s queries as well. Even if two entities are related in some way, if you seldom or never join on the relationship, you can break the relationship to implement the sharding.

• Choosing a partitioning key that lets you avoid cross-shard queries as much as possible but also makes shards small enough that you won’t have problems with disproportionately large chunks of data

Querying across Shards

• Most sharded applications have at least some queries that need to aggregate or join data from multiple shards (for reports)

• Strive to make your queries as simple as possible and contained within one shard.

• For those cases where some cross-shard aggregation is needed, we recommend you make that part of the application logic.

• Cross-shard queries can also benefit from summary tables. You can build them by traversing all the shards and storing the results redundantly on each shard when they’re complete. If duplicating the data on each shard is too wasteful, you can consolidate the summary tables onto another data store so they’re stored only once.

IV. Summary

Optimizing and scaling MySQL is a journey. Before you dive into scalability bottlenecks, make sure you’ve optimized your queries, checked your indexes, and have a solid configuration for MySQL.

Once optimized, focus on determining whether you are read-bound or write-bound, and then consider what strategies work best to solve any immediate issues.

For read-bound workloads, our recommendation is to move to read pools unless replication lag is an impossible problem to overcome. If lag is an issue or if your problem is write-bound, you need to consider sharding as your next step.

That's all about MySQL I know so far. I hope it could help you to consolidate your knowledge and give you some ideas to optimize your MySQL.

Today I will share my experience and what I learned in 3 main topics: Schema Design, Indexing and Query Optimization.

But before we dig dive into them, we should understand MySQL's logical architecture. In other words, we should understand how MySQL processes our queries.

When we send a query to MySQL:

• MySQL will check and open connection (reject if too many connections)
• Parser will parse the query to check the syntax
• Optimizer will optimize the query (based on Performance Schema)
• Run the optimized query

And you see, MySQL will optimize our queries depending on metrics in the Performance Schema before running them. So which data types we use, how we design our tables, how we do the indexing and how we write the queries, will determine the performance of our database. It's hard to read and remember all knowledge so I write it as a quick note with multiple points. Anw, let's start!

I. Schema Design

1. Data Types

Using optimal data types not only reduces the storage space but also improves the query performance (because query data will be loaded onto RAM => if RAM is full, data will be flushed to disk => poor performance)

• Smaller is usually better
• Simple is good (e.g, use decimal rather than string for lat/long values)
• Avoid null if possible: harder for MySQL to optimize queries

Let's check the data types MySQL has and discuss when we should use them:

• Varchar: variable length

  • use 1 or 2 extra bytes to store length, 1 byte if the value requires no more than 255 bytes, and 2 bytes if it’s more

    Ex: varchar(10) will use up to 11 bytes, varchar(500) will use up to 502 bytes

  • Should use varchar(255) to maximize the use of a column because it needs only 1 extra byte for length.
• Char: fixed length of characters => good choice for storing hashed user password
• Text: string data types designed to store large amounts of data, cannot index full length. It will be stored off the table when the value is greater than 8kb and needs to be read from the disk when querying.
• Blob: store binary data (images, videos,...) => in practice, we should save images and videos in other storage space and store image paths in MySQL (avoid full RAM)
• Integer: tinyint 1 byte, small int 2 bytes, int 4 bytes, bigint 8 bytes => the width for integer types such as int(11) which is meaningless for most applications because it does not restrict the range of values
• Real number: Float 4 bytes, Double 8 bytes. Double has greater precision than Float
• Datetime: 1000 - 9999, 8 bytes
• Timestamp: 4 bytes, range 1970-2038, timestamp values will be converted from the current time zone to UTC for storage, and converted back from UTC to the current time zone for retrieval
• Decimal: store numbers with fractional part (e.g, financial data)
• Enum: cannot update value orders in enum, will be stored as int (should use tinyint instead so that you do not need to alter)

Notes: with varchar, text type, we should create a prefixed index if needed.

2. Schema Design Mistakes

• Too many columns: heavy bin log could lead to replication lag, poor performance when pulling data onto memory for read/join queries,...
• Too many joins: should join on 3 tables at most (harder for query optimization)
• Should not use null

II. Indexing

The index is a data structure that storage engines use to find rows quickly. Index performance can drop very quickly when our dataset grows.

1. Types of Indexes

a. B-tree indexes

• A data structure that store data in its node in sorted order
• Leaf pages contain a link to the next for fast range traversals through nodes (has order)
• Leaf pages have pointers to the indexed data
• Storage engine does not have to scan the whole table to find the desired data

Advantages

• B-trees store indexed columns in order => useful for range searching
• Work well for lookups by the full key value, a key range, a key prefix
• Store replated values close together
• Index stores a copy of values => some queries can be satisfied from the index alone (covering index)

=> Index reduces the amount of data the server has to examine

=> Index helps the server avoid sorting and temporary tables

Limitation

• Can’t skip columns in the index

• Can’t optimize accesses with any columns to the right of the first range condition

  VD: where last_name = “Smith” and first_name like ‘J%’ and dob = …. 
  
  Because the LIKE is a range condition so MySQL cannot apply index for **dob** column searching.
  

Btree works with the following kinds of queries:

• Match on the full key value specifies values for all columns in the index
• Match a leftmost prefix: uses only the first column in the index
• Match a column prefix: uses only the first column in the index.
• Match one part exactly and match a range on another part
• Index-only queries: access only the index, not the row storage

b. Hash indexes

Hash indexes use hash tables to store data and have somewhat different characteristics from those just discussed:

• They are used only for equality comparisons that use the = or != operators (but are very fast). They are not used for comparison operators such as < that find a range of values. Systems that rely on this type of single-value lookup are known as “key-value stores”; to use MySQL for such applications, use hash indexes wherever possible.

• The optimizer cannot use a hash index to speed up ORDER BY operations. (This type of index cannot be used to search for the next entry in order.)

• MySQL cannot determine approximately how many rows there are between two values (this is used by the range optimizer to decide which index to use).

• Only whole keys can be used to search for a row. (With a B-tree index, any leftmost prefix of the key can be used to find rows.)

c. Adaptive Hash indexes

The InnoDB storage engine has a special feature called adaptive hash indexes. When InnoDB notices that some index values are being accessed very frequently, it builds a hash index for them in memory on top of B-tree indexes. This gives its B-tree indexes some properties of hash indexes, such as very fast hashed lookups. This process is completely automatic, and you can’t control or configure it, although you can disable the adaptive hash index altogether.

2. Indexing Strategies

a. Prefix

• Selectivity: ratio of the number of distinctly indexed values to the total number of rows in the table => higher is better
• With varchar /Text column: must define prefix indexes => long enough to give good selectivity and short enough to save space
• Cannot use prefix indexes for ORDER BY or GROUP BY queries, nor can it use them as covering indexes

b. MultiColumn indexes

• You need a single index with all relevant columns (AND), not multiple indexes that have to be combined
• OR condition: sometimes use lots of CPU and memory resources to merge
• Choose the most selective columns first in the index (examine the distribution of values in the table, count distinct / count *)

c. Clustered Indexes

• Store the Btree index and the rows together in the same structure
• When a table has a clustered index, its rows are actually stored in the index’s leaf pages

d. Secondary Indexes

• Secondary index accesses require two index lookups instead of one
• A leaf node doesn’t store a pointer to the referenced row’s physical location; rather, it stores the row’s primary key values
• To find a row from a secondary index, the storage engine first finds the leaf node in the secondary index and then uses the primary key values stored there to navigate the primary key and find the row (traverse two Btrees)

e. Covering Indexes

• An index that contains all data needed for a query is called a covering index
• Databases do not need to access the filesystem

III. Query Optimization

1. Optimize Data Access

a. Are You Asking the Database for Data You Don’t Need?

You should find out whether your application is retrieving more data than you need

• Fetching more rows than needed => need limit
• Fetching all columns from a multi-table join => define what columns you need
• Fetching all columns=> cannot use covering indexes, add more IO, memory
  • Avoid using select * from
  • But in some cases, we should query full objects, cache them, and use them many times could increase performance
• Fetching the same data repeatedly => caching

b. Is MySQL Examining Too Much Data?

In MySQL, the simplest query cost metrics are:

• Response time: Service time + Queue time
• Number of rows examined
• Number of rows returned

Ideally, the number of rows examined would be the same as the number returned (100%). To reduce the number of examined rows:

• Use covering indexes: no need to retrieve rows from tables
• Change the schema, using summary tables (prepare summary/report tables in advance)
• Rewrite a complicated query so the MySQL optimizer is able to execute it optimally

2. Ways to Restructure Queries

a. Complex Queries Versus Many Queries

The traditional approach to database design emphasizes doing as much work as possible with as few queries as possible. This approach was historically better because of the cost of network communication and the overhead of the query parsing and optimization stages.

However, this advice doesn’t apply as much to MySQL because it was designed to handle connecting and disconnecting very efficiently and to respond to small, simple queries very quickly. Modern networks are also significantly faster than they used to be, reducing network latency. So running multiple queries isn’t necessarily such a bad thing.

It’s still a good idea to use as few queries as possible, but sometimes you can make a query more efficient by decomposing it and executing a few simple queries instead of one complex one.

b. Chopping up a query

• Need to delete old data => chop up a DELETE statement and run sequentially

   E.g: DELETE FROM messages WHERE created < DATE_SUB(NOW(),INTERVAL 3 MONTH);
  
      => we will limit the number of affected rows:
  
   DELETE FROM messages WHERE created < DATE_SUB(NOW(),INTERVAL 3    MONTH) LIMIT 10000
  

=> minimize the impact on the server (smaller transactions), reduce replication lag

=> it might be a good idea to add some sleep time between DELETE statements to reduce the load on servers.

c. Join Decomposition

• MySQL can run well over 100000 simple queries per second
• Moderns networks are also significantly faster than they used to be

=> sometimes you can make a query more efficient by decomposing it and executing a few simple queries instead of one complex one.

Advantages

• Caching can be more efficient
• Reduce lock contention
• Doing joins in the application makes it easier to scale the database by placing tables on different servers (because we do not need to join)
• The queries themselves can be more efficient (such as using an IN() instead of a join lets MySQL retrieve rows more optimally)
• You can reduce redundant row accesses. Doing a join in the application means you retrieve each row only once, whereas a join in the query is essentially a denormalization that might repeatedly access the same data

Disadvantages

• Need to perform the join in the application
• Require a certain level of your team

IV. Conclusion

Phewww, we've just discussed about 3 main and most important topics in MySQL. I can not show all my experience and knowledge about MySQL in one post, so this post is like a note for me or anyone else to review and recall tips and strategies to improve MySQL performance.

In the next post, we will discuss Replication and Scaling techniques in MySQL. I will share how I scaled my database and improved read/write performance.

See you next time!

References

1. High Performance MySQL: Optimization, Backups, and Replication
2. MySQL 8.0 Documentation

I. Preparation

A cache always has 2 main functions: get and put. And the requirements here are get and put method must run in O(1) average time complexity.

=> we will store data in a Hashmap.

we invalidate and remove the least frequently used key. When there are 2 or more keys with the same frequency, we will remove the least recently used key.

=> we will store the frequency of keys in a Hashmap <Frequency, LinkedList of Key> and will maintain the least recently used by the LinkedList of keys in the value.

=> we also need to maintain what is the min frequency of cache, so when you update the freqMap, you also need to update the min frequency.

And this is my note before I implement:

• class LFUCache:

  • capacity
  • min freq
  • freqMap: map of (freq, double linked list of nodes)
  • cache: map of (key, node)

• get:

  • if not exist: return -1
  • if exist:
    • check if the node has freq = min_freq and list size == 0 => min_freq++ (because after that we will increase cur freq by 1)
    • increase node frequency by 1
    • remove the node from the current LinkedList in freqMap and insert it to the head of the list at freq + 1 key
    • return value

• put:
  • if exist: update the value of the key in the cache, increase the freq of the node by 1, and update freqMap with the new freq.
  • else:
    • full capacity: get linked list from freqMap by min freq of cache, remove the last node in freqMap and cache.
    • create a new node with freq = 1, reset min freq to 1
    • insert to freqMap: get the cur list or create a new list if not exist, add the new one, and put the list again.
    • insert into cache.

II. Implementation

Phewwww, here is my implementation:

import java.util.HashMap;
import java.util.Map;

class LFUCache {

  private int capacity;
  private int minFreq;
  private Map<Integer, DoubleLinkedList> freqMap = new HashMap<>();
  private Map<Integer, Node> cache = new HashMap<>();

  public LFUCache(int capacity) {
    this.capacity = capacity;
  }

  public int get(int key) {
    if (!cache.containsKey(key)) return -1;
    Node node = cache.get(key);
    updateFreqList(node);
    return node.val;
  }

  public void put(int key, int value) {
    if (capacity == 0) return;
    // update value and frequency when exist
    if (cache.containsKey(key)) {
      Node node = cache.get(key);
      node.val = value;
      updateFreqList(node);
    } else {
      // remove when cache is full
      if (cache.size() == capacity) {
        DoubleLinkedList minFreqList = freqMap.get(minFreq);
        Node minNode = minFreqList.removeLast();
        cache.remove(minNode.key);
      }

      Node newNode = new Node(key, value);
      minFreq = 1;
      DoubleLinkedList curList = freqMap.getOrDefault(1, new DoubleLinkedList());
      curList.add(newNode);
      freqMap.put(1, curList);
      cache.put(key, newNode);
    }
  }

  private void updateFreqList(Node node) {
    // remove node from cur list
    DoubleLinkedList curList = freqMap.get(node.frequency);
    curList.remove(node);

    if (node.frequency == minFreq && curList.size == 0) minFreq++;

    node.frequency++;
    // insert into new list with new freq
    DoubleLinkedList newList = freqMap.getOrDefault(node.frequency, new DoubleLinkedList());
    newList.add(node);
    freqMap.put(node.frequency, newList);
  }

  class Node {
    int key;
    int val;
    Node next;
    Node prev;
    int frequency;

    public Node(int k, int v) {
      key = k;
      val = v;
      frequency = 1;
    }
  }

  class DoubleLinkedList {
    Node head;
    Node tail;
    int size;

    public DoubleLinkedList() {
      this.size = 0;
      head = new Node(0, 0);
      tail = new Node(0, 0);
      head.next = tail;
      tail.prev = head;
    }

    public void add(Node node) {
      node.next = head.next;
      head.next.prev = node;
      node.prev = head;
      head.next = node;
      size++;
    }

    public void remove(Node node) {
      node.prev.next = node.next;
      node.next.prev = node.prev;
      size--;
    }

    public Node removeLast() {
      if (size > 0) {
        Node tailNode = tail.prev;
        remove(tailNode);
        return tailNode;
      }
      return null;
    }
  }
}

III. Result

Phewwwww, it's done. I implemented my own doubly linked list, and compare it with when I use LinkedList of Java Core. Surprisingly, my own linked list had better runtime (149ms) while Java core's LinkedList triple (509ms). That may be because the core LinkedList needs to do many other actions (when we add and remove) rather than focusing on only this problem =)))

Okay, that's all for today's post.

IV. Enjoy your weekend

By the way, let's see =))). Autumn has already come. Go out and enjoy now =)))

Enjoy your weekend!

I. CORS

• What: Technique that allows a server to indicate any other domains which can make requests to that server in the browser

• Why we need: the same origin policy of browser which restricts JavaScript code from making requests from one domain to another domain

  Ex: you open a Facebook tab and a hacker website tab on your browser. The tab Facebook uses JS to request to the server, if there is no same origin policy, JS of hacker website could also make requests to the Facebook server. That’s why the browser needs to have a policy to detect JS of a resource could access other resources or not

• How to work:

  • a client send a request to a server with Origin header which contains the domain of the current website
  • The server will validate the Origin, and if valid, return a response with the header Access-Control-Allow-Origin (often have the same value as the Origin header)
  • If there is no Access-Control-Allow-Origin header in a response or an invalid value there, the browser will return an error

• Config CORS in Spring Boot:

  • Use @CrossOrigin annotation to enable cross-origin calls from a list of other domains
  • Define maxAge to cache the preflight response
  • Config at method level, class level, or globally
  • To config globally, you need to config a configuration bean that implement webMvcConfiguer to add CorsMappings

II. Attack Basics

1. SQL Injection

• What: Attack that uses malicious SQL statement to access information that is not intended to display.

• Results:

  • Extract sensitive data
  • Delete data or drop tables

• How to prevent:

  • do not use string concat to build a query
  • use parameterized queries with spring JPA or JDBC to prevent that.
  • Principle of least privilege: reduce the permissions of the application/users at runtime
    so it can at most edit data, but not change table structures
  • Password hashing: use one-way hash algorithms such as Bcrypt, can use with salt

2. XSS - Cross-Site Scripting

Attackers can inject malicious JavaScript into your website.

a. Persistent XSS

• How: user can add content such as new comments, and those comments which contain malicious JavaScript will be stored in database. So any other users who see that comment will be attacked, and their browser will run this javascript

• Results:

  • Spreading malicious js on social media sites, auto download something to your computer,...
  • Steal your session
  • Steal your sensitive data

• How to prevent:

  • Escape dynamic content: replace significant characters with HTML entity encoding => the script will never be treated as executable code by the browser
  • Whitelist values: users have to select from a list rather than provide any input
  • Implement Content-Security Policy that defines script source and tells the browser to never execute inline JavaScript
  • HTTP-only cookies: mark cookies as HTTP-only so cookies will be received, stored, and sent by the browser only, but cannot be modified or read by JavaScript

• Set HTTP-only on Spring:

  • Create a ResponseCookie object
  • Set HTTP only = true, set maxAge
  • Add cookie to the response

b. Reflected XSS

• How:

  • Hacker injects malicious JavaScript into the query string and sends it to users
  • Users click to that URL, the server will respond the parameter back to the user (such as on the search results page)
  • Then the browser will render the script (send requests to the hacker website with the param is the cookie of users).
  • Hacker server has session or cookie, then can call to that website as that user.

• Types of pages will be attacked:

  • Search results: search criteria get displayed back to the users
  • Error pages: have an error message which contains invalid input, does the input get escaped properly when it is displayed back to the user?
  • Form submissions: if a page post data, does any part of the data being submitted by the form get displayed back to the user?

3. CSRF

• What: If hackers can forge HTTP requests to your site, they may be able to trick your users into triggering unintended actions

• How to attack:

  • User login to their website A
  • Hacker create a malicious website and trick users to click on their link
  • In the hacker’s website, there is a form or img to forge a request to website A with user’s cookie
  • Website A has no way to distinguish between a forged request and an actual request
  • That request could do some intended action via the user’s account such as posting a new post, spreading worms on social media, or transferring funds from the visitor’s account to others,...

• How to prevent:

  • Use REST standard: GET requests are used only to view resources => limit the harm that can be done by malicious URLs - an attacker will have to work much harder to generate a harmful POST request

  • Client-side generated CSRF-tokens: client code generates and sends the same unique secret value in both Cookie and a custom HTTP header.

    => considering a website is only allowed to read/write a Cookie for its own domain, so only a real website can send the same value in both headers

    => The server will compare the token attached to the request with the value stored in the cookie.

  • Check the HTTP Referer and Origin header: check the expected domain which triggered the request and reject any requests with abnormal domains

4. DDOS

• What: Denial of Service, when attackers attempt to make your website unavailable to others
• How: flooding with requests to exhaust all the available resources (server resources, network bandwidth) => real users are unable to get access
• Types:
  • SYN flood: exploits 3-way handshake of TCP when client does not send ACK message to server so may connections do not close so that the server cannot open new connections with real users
  • HTTP Flood: hackers will exploit legitimate GET or POST requests => exhaust the server’s connection pool
  • Others: ...
• How to prevent:
  • Block IP address => hackers will use distributed DOS
  • Apply rate limit for IP address, users
  • Autoscaling
  • Caching commonly accessed resources to reduce database access
  • Serve your images, videos and other resources from CDN so that you are offloading accessed resources to a third-party service designed to withstand large amounts of traffic

III. Other Concepts

Hash vs Encrypt vs Encode

• Hashing is the act of transforming data from arbitrary size to a fixed size value. => use for checking the integrity of data or verifying the password (Bcrypt hashing)
• Encode is a way to transform data into another form to preserve usability. Ex: we transform text, audio, image into bits 1 and 0 so computers can understand, store and process them.
• Encryption is the act of transforming data into another form to preserve the confidentiality

  Ex: RSA encryption uses a public key to encrypt and a private key to decrypt data

Is there any way to crack Hash?

Yes of course. We can try to guess the original string by calculating the hash of every possible input and then comparing the results.

Encryption Algorithms

• Symmetric encryption: uses the same key for both encrypting and decrypting data (such as AES,...)
• Asymmetric encryption: uses a public key to encrypt and a private key to decrypt data (such as RSA,...)

To be continued...

Tùy vào mission thế nào, tự khắc bạn sẽ biết mình cần làm gì. Và bao giờ cũng thế, khi mục tiêu của bạn đã rõ, in ra dán đc lên trán, thì cũng là lúc bạn nhận thức được việc gì quan trọng hay không quan trọng, học cái này hay cái kia, và học để dùng cho việc gì.

Đặt mục tiêu càng sớm, bạn càng rút ngắn được thời gian hoàn thành mục tiêu, không sa đà vào những thứ không quan trọng. Đương nhiên là phải có kỷ luật nữa.

Tôi xác định mục tiêu khá muộn so với bạn bè đồng trang lứa. Tôi cũng không hối hận vì đổi lại là tuổi trẻ nhiều sắc màu cung bậc. Đó là quãng thời gian sống với âm nhạc, những chuyến đi dài rong ruổi, gặp những con người thú vị và lắng nghe những câu chuyện hay. Đến một ngày, tôi dừng lại và tự hỏi mình sẽ làm gì tiếp đây. Và cho đến khi đã ra trường khoảng hơn 1 năm, tôi mới có câu trả lời.

Và từ đó đến nay, tôi luôn nỗ lực cho những việc quan trọng, liên quan trực tiếp đến mục tiêu. Đặc biệt là tôi luôn cân nhắc việc có nên học 1 thứ gì đó không, hay tôi có thực sự cần nó trong công việc hay cuộc sống không. Bởi vì học mà không dùng thì quá lãng phí thời gian. Cũng giống như hàng đống kiến thực bạn được nhồi nhét trên trường, tới giờ bạn còn nhớ hay áp dụng được những gì???

Tới giờ đi làm rồi, có ai quan tâm vòng Benzen ra răng? hay các hàm tích phân tầng tầng lớp lớp dùng cho việc chi? (Trừ những bạn nghiên cứu đúng chuyên ngành thì mình k nói).

Cho nên khi một ai đó khen một thứ gì đó hay và bảo tôi học đi, tôi luôn nghĩ ngay tới việc sẽ áp dụng nó vào đâu, mức độ ưu tiên, trước khi quyết định dành thời gian cho nó.

Ngành IT kiến thức là vô biên, nếu không biết chọn lọc thì thực sự bạn dễ bị "cái gì cũng biết mà đếch biết cái gì". Như cá nhân tôi đã xác định trong 3 năm tới chỉ tập trung làm Backend, thì tôi phải dành tới 95% nguồn lực cho nó. Hôm trước, trong buổi phỏng vấn vị trí Backend Dev, tôi được hỏi nếu khách hàng yêu cầu thì em có sẵn sàng chuyển qua code Front-end không? Ngay lập tức tôi chia sẻ luôn định hướng của bản thân và bày tỏ không sẵn sàng làm như thế. Vì tôi biết nếu chạy theo những gì khách hàng yêu cầu thì tôi sẽ bị xoay như chong chóng, lâu dần chẳng biết mình mạnh cái gì.

Xã hội ngày càng phân công rõ ràng. Các công ty đang có xu hướng tuyển người chuyên về một lĩnh vực nhất định chứ không đỏi hỏi một người biết mọi thứ nhưng chẳng thứ gì sâu sắc. Tốt nhất bạn nên chia sẻ định hướng rõ ràng với nhà tuyển dụng để tìm được một môi trường phù hợp.

Quay lại chuyện học gì và học để làm gì, chỉ có bạn mới trả lời được, sau khi đã biết mission của mình. Tuy nhiên hãy nhớ: phần lớn kiến thức mà bạn đã học trên trường chẳng áp dụng được cho cuộc sống hiện tại của bạn. Do vậy lựa mà học thứ cần thiết thôi :))) Còn ti tỉ thứ quan trọng đang chờ bạn khám phá, với điều kiện bạn phải nhận ra chúng trước. Khi đã nhận ra rồi thì cắm đầu mà chạy tới đích chứ còn ngó nghiêng làm gì nữa :)))

02-08-2022 23:05

Duy Hieu

• Where: Applicable for orders with pickup and delivery addresses within the urban area in Hanoi and Ho Chi Minh City.

• Who: shops and personal customers with fruits, food, clothes, cosmetics, books, components,...

• Time: 7h-21h30

XFAST commits that the maximum delivery time of orders will be specifically applied to each kilometer as follows:

• 0 - 3km delivered in 30 minutes

• 3 - 6km delivered in 60 minutes

• 6 - 9km delivered in 90 minutes

• More than 9km delivery only from 2-3 hours

Cost:

• 0.9 USD for a package with distance <= 2km
• plus 0.15USD/km if distance > 2km
• order from 18h-6h: additional cost 0.2USD

For the best support, Shop can contact the following channels to answer questions from GHTK:

– Email: cskh@ghtk.vn

– Fanpage: Giaohangtietkiem.vn

I. Exception handling is not supposed to replace a simple test.

As an example of this, we wrote some code that tries 10,000,000 times to pop an empty stack. It first does this by finding out whether the stack is empty.

if (!stack.isEmpty()) s.pop();

Next, we force it to pop the stack no matter what and then catch the EmptyStackException.

try {
   stack.pop();
}  catch (EmptyStackException e) {
}

On my test laptop, the version that calls isEmpty ran in 20 milliseconds. The version that catches the EmptyStackException ran in 3640 milliseconds.

As you can see, it took far longer to catch an exception than to perform a simple test. The moral is: Use exceptions for exceptional circumstances only.

II. Do not micromanage exceptions.

Many programmers wrap every statement in a separate try block.

for (i = 0; i < 100; i++) {
      try {
        this.method1()
      } catch (EmptyStackException e) {
        // problem 1
      }
      try {
        this.method2()
      } catch (IOException e) {
        // problem 2
      }
}

This approach blows up your code dramatically. Just wrap the entire task in a try block. If any one operation fails, you can then abandon the task.

for (i = 0; i < 100; i++) {
      try {
        this.method1()
        this.method2()
      } catch (EmptyStackException e) {
        // problem 1
      } catch (IOException e) {
        // problem 2
      }
}

This code looks much cleaner. It fulfills one of the promises of exception handling: to separate normal processing from error handling.

III. Make good use of the exception hierarchy.

Don’t just throw a RuntimeException. Find an appropriate subclass or create your own. Don’t just catch Throwable. It makes your code hard to read and maintain.

Do not hesitate to turn an exception into another exception that is more appropriate. For example, when you parse an integer in a file, catch the NumberFormatException and turn it into a subclass of IOException or FileInputFormatException that you declared.

IV. Throw early, catch late

Some programmers worry about throwing exceptions when they detect errors. Maybe it would be better to return a dummy value rather than throw an exception when a method is called with invalid parameters?

For example, should Stack.pop return null, or throw an exception when a stack is empty? We think it is better to throw an EmptyStackException at the point of failure than to have a NullPointerException occur at later time.

Many programmers feel compelled to catch all exceptions that are thrown. Often, it is actually better to propagate the exception instead of catching it:

Higher-level methods are often better equipped to inform the user of errors or to abandon unsuccessful commands.

(Source: Core Java Volumn I - Fundamentals)