Google PubSub in 6minutes

Here’s a guide to quickly learn Google Pubsub and deploy & configure your microservices to run on it.

What is PubSub?

PubSub, short for Publish-Subscribe, is a way to create asynchronous communication between microservices.

With Pubsub, we have a publisher application and a subscriber application. The publisher application creates a message and sends it to Cloud PubSub. Then, PubSub stores the message until it reaches to all subscribers.

The communication between the services can be one to many (fan-out), many to one(fan-in) or many to many. It’s great for IoT systems, System Monitoring, database backup and log management as well as programming.

Why PubSub?

PubSub provides the benefits such as:

• Separated Communication and Application logic; hence providing modularized and secure projects with higher quality code.
• Instant delivery of messages to all subscribers, regardless of the location. No polling, no checking for messages!
• Since it’s not dependent on any programming language it can be easily integrated to anywhere, making the development process simpler.
• You can add as many publishers or subscribers as you’d like. It’s highly scalable.
• Since the communication and the logic are two separate layers, it’s easier to troubleshoot.

Getting Started

To begin with integrating PubSub, first, you must make sure to sign in or create a Google Cloud account. Then, simply navigate to this link. Now let’s complete the following steps to complete our first integtration:

• On the navbar to the page, either select an already existing project for integration or create a new one. For the sake of this tutorial, I’m going to create a new one by clicking project-name / New Project. Then give my project a name and click Create. Then, select the newly created project from up in the navbar.
• Click on the Menu icon and then View All Products. Click on PubSub. You can pin it to your menu by clicking on the pin icon to the left of it.
• Click on the Create Topic button. Give it a name and click Create. Then scroll down to the Create Subscription button and click it. Give the subscription a name. Keep everything default and click Create. To create a one to many system, simply create another subscription following the same steps.
• Now in order to control and limit access, we must set up a service account for our service and grant IAM permissions. Simply click on IAM & Admin and then Service Accounts on the main menu to the left. Then, click Create Service Account button. Give it a name and click Create. Select Pub/Sub Publisher and Pub/Sub Subscriber for permissions and click Continue, and click Create.
• Click onto the three vertical dots to the right side of your newly created Service and click Manage Keys. Then navigate to Add Key, Create new key and select JSON. Download the file and name it key.json. Move it to project folder directly under the project-name/.
• Now type Cloud Build API and Cloud Run API onto the search bar and enable both. (You may be asked to enable billing if you haven’t already done that, so do enable. It won’t charge you for your trial phase. No worries!)

Although the PubSub can be used for any project regardless of the language, I’m going to show you an example on NodeJs. You can adapt the steps below to whatever your project is. Let’s first start the publisher service:

• Make sure you’ve got the Google Cloud SDK first. You can install it by clicking here and to ensure you’ve got it on your system simply open the terminal and run the command “gcloud”.
• Install the PubSub client library based on the framework/language your project uses. For NodeJs, try this library.
• Then, we need to dockerize our project. For the NodeJs and related frameworks, you can learn how to dockerize your application by clicking here.
• Go back to the cloud platform and search for Billing Projects, then find the one you just created and copy the Id. Then open the terminal, navigate to your project folder. For each microservice simply go under the microservice directory and run the command (change the project id to what you’ve copied):

gcloud builds submit — tag gcr.io/PROJECT-ID/publisher-microservice-name

• After the command completes it’s job, go back to Google Cloud and type Conrainer Registry to the search bar to ensure that the docker container is pushed successfully.
• Click on the three vertical dot icon that is to the right of your container then, and select Deploy to Cloud Run. Everything there will be filled in automatically so unless there’s some custom configuration you’d like to make, simly click Next. The only thing you need to watch out for is that you must enable non GCP users to access the API by clicking Allow unauthenticated invocations, because we want to set up the authentication inside out app not via the GCP. Then, you may click Create. Your container is now deployed, and the URL to acces that will be given to you on the page that is opened automatically. You can make sure it works by sending a request to that url via postman.
• If you change anything regarding your project, go to Cloud Run on Google Cloud and click on Edit & Deploy New Revision. Then, without needing to change anything, you can simply click Deploy.

Now we can move forward to the subscriber service. Simply create another application to consume the data from the publisher by following the steps below:

• Copy the files .dockerignore, Dockerfile and .gcloudignore to your service to the project level. You won’t need the key.json for the subscriber.
• Navigate to the subscriber project directory via the terminal and tun the command:

gcloud builds submit — tag gcr.io/PROJECT-ID/subscriber-microservice-name

• After the command completes it’s job, go back to Google Cloud and type Conrainer Registry to the search bar to ensure that the docker container is pushed successfully.
• Click on the three vertical dot icon that is to the right of your container then, and select Deploy to Cloud Run. Everything there will be filled in automatically so unless there’s some custom configuration you’d like to make, simly click Next. Unlike before at this point you must check the Require authentication with the subscriber service for security. Then, you may click Create. Your container is now deployed, and the URL to acces that will be given to you on the page that is opened automatically. You can make sure it works by sending a request to that url via Postman.
• If you change anything regarding your project, go to Cloud Run on Google Cloud and click on Edit & Deploy New Revision. Then, without needing to change anything, you can simply click Deploy.
• Now we need to create another service account to invoke Cloud Run. To do so, click on IAM & Admin and then Service Accounts on the main menu to the left as before. Then, click Create Service Account button. Give it a name and click Create.
• Log in to your Cloud account via the terminal, select your project and run the following commands to grant permissions for your subscriber:

GOOGLE_CLOUD_PROJECT=project-id

PROJECT_NUMBER=project-number

gcloud run add-iam-policy-binding microserviceName — member=serviceAccountName:pubsub — run — invoker@$GOOGLE_CLOUD_PROJECT.iam.gserviceaccount.com — role=roles/run.invoker — region us-centrall — platform m

Note: On the last command, make sure to change the microServiceName and the serviceAccountName to your own.

• Go to Cloud Run again, find your subscriber service account and click on the button Add Trigger. Give it a name and select Cloud Pub/Sub topic as an event. Select your publisher topic as the topic right below that. Select Pub/Sub Cloud Run Invoker as the service account. Click Save.

And that’ll be all! You can now test if it works via Postman and start using the microservices. You can add as many publishers and subscribers as you’d like repeating the steps above.