CST438 - Week 8

CST438: Software Engineering – Week 8

This the final week of CST438 Software Engineering course. I have learned so much from the labs, assignments, module lectures, and the textbook “Software Engineering at Google” by Winters, Mantric and Wright. The 5 most important things that you learned in the course are:

1- Coding of a Spring Boot REST backend application.
2- React frontend application.
3- SDLC (software development life cycle) phases
4- Coding Selenium system test
5- Write full software requirements specifications (SRS).

The concept of REST (REpresentational State Transfer) was a great way to expand my knowledge of backend architecture. It was interesting to see how simple it is to exchange and manipulate data using HTTP methods (GET, POST, DELETE, PUT). The course also introduced Spring server and the use of JPA queries. The first lab was a hands-on REST project where we built a project from scratch using Spring Boot. By the end of the lab, we had created REST controllers, DTO classes, entity classes, and repository interfaces to experience a full working backend portal.

The frontend was built with React, which supports component-based development and optimizes the DOM. This was my first time working with React, and I learned how to use ES6 syntax to write HTML inside JavaScript. The course covered many core React concepts such as UI rendering, passing data with props, using hooks like useState, and handling events with controlled inputs.

Another important topic in the course was the Software Development Life Cycle (SDLC). It was helpful to study the different processes involved in producing and releasing software to a client. Our team project followed the Agile process to deliver the backend, frontend, unit tests, and system tests in a realistic software project setting. I also got valuable experience with team coding and code reviews using GitHub over three weeks.

For system testing, we used Selenium to automate web browsers. I learned how to navigate the browser automatically and use data to test frontend behavior. It was exciting to work with such a powerful tool that saves time and integrates well with other frameworks.

Finally, practicing how to write Software Requirements Specifications (SRS) was essential for learning how to gather requirements before coding. It gave me a solid understanding of IEEE SRS standards and helped me build the communication skills needed for working in the industry.

CST438 - Week 7

CST438: Software Engineering – Week 7

Agile vs Waterfall

In week 7, we are exploring the differences between Agile and Waterfall methods in software development. When we talk about Agile, most requirements come from user stories and their continuous feedback during each iteration. In contrast, Waterfall relies heavily on documentation created early on to outline software specifications and requirements. The requirements gathering process involves formal documentation of use case scenarios, user interviews, and reviews of similar platforms when applicable. Making changes later in the development cycle can be difficult with Waterfall, so change management is controlled by CAB stakeholders who must follow strict procedures to approve or deny changes. Unlike Waterfall, Agile allows changes during the iterative process to accommodate mandatory design prerequisites or simple customer requests.

Waterfall methods include cost estimation as part of the initial requirements phase, resulting in a planned design that strictly follows those requirements. It’s important to note that Waterfall follows a linear progression through phases like requirements gathering, design, implementation, testing, and maintenance. The advantages of following a documented design with a scheduled plan include better project predictability, easier progress tracking, clear milestones, and improved control over scope and budget. Agile flexible approach to changes, however, can delay projects and drive up software development costs. Ultimately, the choice between Agile and Waterfall depends on the needs and priorities of stakeholders, developers, and the project context.

CST438 - Week 6

 

CST438: Software Engineering – Week 6

We are approaching the final weeks of the Software Engineering course. The team has been working on the project for the third iteration of system testing and is making good progress. We look forward to finalizing the portal and publishing it to Amazon AWS in the coming week. This week focused on hands-on coding with Selenium libraries to automate the testing and verification of data entries in the basic learning management system (LMS). The process of software engineering and interaction with team members during this project was extremely helpful in gaining experience with team coding. As we progress, periodic communication with team members is essential for the project's success. Using the GitHub organization model allowed the project's progress to be shared among the team members as tasks advanced through each stage.

The reading for this week focuses on computers as a service and how Google scaled its infrastructure management by adapting to computing automation and embracing containers as resources. Google runs two types of jobs in its environment: batch jobs and serving jobs. The batch jobs are short-lived and easy to start, primarily focusing on the process. The serving jobs are mainly stateful, longer-running jobs, such as IIS services, load balancers, and web services. The serving jobs focus on full loads, redundancy, and latency. One of the challenges Google faced using containers is the locality of critical state. For instance, losing a container means losing its local state, such as a database or storage. The solution to this problem is to move all critical states to external resources, while leaving non-critical and local state recreation intact. In other words, containers become recyclables for non-critical states. How about batch jobs contingent on the local state? Google uses more frequent checkpoints for batch jobs to reduce the likelihood of loss. Additionally, caches are refreshed and are not an issue if lost.

Abstraction in design is necessary not only for scalability but also provides a layer of protection against indirect internal changes. For instance, Hyrum's law exposes the unexpected use of the PID range in a leaky abstraction design. PID max changes lead to log system breaks and naming collisions. Over time, we will need CaaS that provides dynamic abstraction for specific behaviors and withstands rapid changes – containers could help us with that.

Google Borg changed the handling of batch jobs and serving jobs on isolated machines, combining everything under a shared pool. As a result, the unified compute system consolidated tools and administration, lowering the CapEx and OpEx of operations. Borg also provided dynamic workloads by reclaiming resources from batch jobs, overprovisioning unused capacity to serving jobs, or allocating idle computing resources to batch jobs. Although the unified Borg system maintained a robust balance of resources and efficiency, it came at the cost of higher complexity. Moreover, Google made a significant investment and drew on Google's workforce to establish Brog as a robust infrastructure for containers, an option that is not readily visible to many companies.  Alternatively, public cloud offers easy-to-scale, offloading infrastructure management. Hybrid cloud is another flexible option that combines both on-premises and public cloud, extending its resources to the public cloud.

CST438 - Week 5

CST438: Software Engineering – Week 5

We have reached the fifth week of the software engineering course. We are reading about Google's large test and its scope to ensure the system works overall. The book mentions several reasons why we need a larger test and why unit tests are insufficient, potentially leading to undiscovered errors. For example, when using mocks, we replace the real data, and mocks may not behave in the same way. Another issue is that mocks written by someone other than the engineer who wrote the code may not understand how it works. Bad, wrong, and untested configurations are sources of serious bugs at Google. Besides configuration changes, faster than the code itself, causing a mismatch that breaks functions. Unit tests aren't designed to run under simulated real-world traffic that could be high and exceed the code's performance capacity. Therefore, large tests are needed to cover the unit test gaps and catch unexpected behavior.

Although large tests were introduced to Google before unit tests in products like AdWords, Search, and Gmail, unit tests later became the focus. The workflow of a large test involves obtaining the SUT for the system under test, seeding the data, interacting with the system under test, and reviewing the results. In some cases, complex SUT large tests are simulated in the full production environment to yield high fidelity. It is essential to balance the size of the large test by selecting an SUT that satisfies test reliability, the cost associated with running infrastructure, and the speed of the test.

When working with large tests, the system under the test requires seeded data to initialize the test stage, along with test traffic generated by the test when data is sent. Examples of seeded data include domain data, baseline data that resembles activities between uses, and complex data generated by APIs over time. Data could be sampled, copied from production, or manually created. Type of larger tests at Google: 

• Functional testing of one or more binaries 
• Browser and device testing 
• Performance, load, and stress testing 
• Deployment configuration testing 
• Exploratory testing 
• A/B diff (regression) testing 
• User acceptance testing (UAT) 
• Probers and canary analysis 
• Disaster recovery and chaos engineering 
• User evaluation

Authoring larger tests requires clear documentation and examples to standardize the procedure. Google runs larger tests in ways that are as familiar as possible to engineers; however, some tests need detailed instructional documentation. In the meantime, tests must run at an acceptable speed to encourage engineers to run tests more frequently. Larger tests must have clear owners listed, responsible for the test's health communications, and updates. Additionally, owners must be able to support the tests in the case of failures.

On another note, the team met to discuss the current iteration for the project. We had issue with merging conflicts last time, but we were able to determine the source of the issue and corrected it. I have been working on the frontend to implement the assignment view, assignment update, assignment addition, and the grading score recording. I definitely learned a lot about REST control and React interaction with backend, however, the syntax need more practice to master it. That’s my update this week.

CST438 - Week 4

CST438: Software Engineering – Week 4

I find the book Software Engineering at Google a very good choice for this course, since it highlights the journey of Alphabet as one of the largest software development companies. The book goes beyond the short-lived academic coding assignments by exploring the formal stages of development and maintaining sustainable software.

The first chapter lays the fundamental distinction between software engineering and programming. The most interesting part of the book is the details regarding Google’s framework for its codebase and culture. For instance, Google strictly follows a process for code review on any changes. The process is simple, efficient, and effective in ensuring the integrity of the code before committing changes. The use of a scalable critique tool processes review requests in a timely manner in support of larger changes.

Additionally, Google has changed software testing by adopting automation, leading a major transformation in software engineering practices. In my opinion, this cultural shift is one of the most impactful changes Google has made to accelerate the development and deployment of technological innovations. These innovations have influenced modern DevOps culture, agile development, and large-scale engineering, allowing faster and more reliable releases. Many tech companies have now adopted similar models, increasing software release velocity and stability across the industry. It is worth noting that Google did not invent automated testing but refined it to support operations at scale. Finally, the authors share their extraordinary knowledge and experiences from years of working at Google, giving readers a deeper understanding of what it takes to build and scale software at a high level.

CST438 - Week 3

 

CST438: Software Engineering – Week 3

Git is a powerful platform that allows software engineers to manage source code and streamline the development process. By branching copies from the origin master, engineers can track changes and manage activities efficiently. This approach is especially effective for deploying tested code prior to merging it back into the main branch.

Branching strategies are commonly used to release software versions derived from a parent version. Depending on the software, this can result in either an update or an entirely new release. Developers also leverage Git to create isolated development environments in distributed Git for collaboration and version control.

For example, an engineer may pull the code from a remote Git repository to their local machine, make changes, and then push the updated code back to the remote. This workflow runs smoothly as long as no other developer has pushed a newer version in the meantime. If they have, the push will fail due to a mismatch. To avoid this, engineers are encouraged to pull from the remote regularly.

Merge conflicts can still occur, particularly when two or more developers modify the same line of code. Resolving these conflicts requires manual intervention—reviewing and editing the affected lines to determine which changes to keep.

Another key feature Git offers is the pull request. This enables team members to review each other’s code before it is merged into the master branch. Once the review is approved, the pull request is closed and the changes are merged.

 

CST438 - Week 2

 

CST438: Software Engineering – Week 2

Mocks are used to eliminate dependencies in testing, enabling fast, independent, repeatable, and timely unit tests. They are powerful tools that allow developers to avoid setting up full configurations for databases or external services when running JUnit tests. This lets developers focus on testing the behavior of specific functions without fully creating databases or external data sources for each test.

For example, if a test depends on other components to run first—such as initializing variables or creating fields—running the test in isolation can be difficult. JUnit mocking solves this by providing predefined responses from those dependent components, ensuring they don't interfere with the current test.