Spring Boot: Uploading and download a file from GCP Storage (Google Cloud Storage)

Spring Boot: Uploading and download a file from GCP Storage (Google Cloud Storage)

Hello fellow developer, if you have landed on this page by search that means my SEO is working and google official documentation is not yet updated for the new approach. or if you have visited just to learn let me add few more details about GCP and spring boot.

Google Cloud Platform or GCP is another cloud provider just like AWS or AZURE, and like most of the cloud provider GCP(Google Cloud Platform) also has the Object Storage, its just Named "Google Cloud Storage". Object store is mechanism of storing your file as object may be in string or some other serialized mechanism, if treated as object a part the file can be stored in different location, or chunks of file can be duplicated which open infinite advantages of scalability, Due to its highly scalable nature the cost to the end user is also less hence the wide adaption.

And now spring boot, its an well maintained and highly popular web framework, almost more than 50% of new java  web projects are considered  to go with Spring boot. it just like spring framework  where the bootstrapping the application is already done in starter dependency and its modular so necessary dependencies can be plugged in. So now Lets dive into the Context.

Before starting of with the coding create the storage bucket in GCP console. and download the credentials Jason using this link. [ https://console.cloud.google.com/apis/credentials/serviceaccountkey?_ga=2.267088910.1279032308.1612845529-1355217530.1612536668 ] and add the key `GOOGLE_APPLICATION_CREDENTIALS` containing the path of the json file you downloaded, Usually I add it in my ide so it can be change frequently.

  1. Dependency

I have actually pasted the complete pom of the sample application. so main requirement is google client library

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<parent>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-parent</artifactId>
		<version>2.4.2</version>
		<relativePath/> <!-- lookup parent from repository -->
	</parent>
	<groupId>com.ashrithgn</groupId>
	<artifactId>google_sotrage_example</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<name>google_sotrage_example</name>
	<description>Demo project for Spring Boot</description>
	<properties>
		<java.version>15</java.version>
	</properties>
	<dependencies>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-web</artifactId>
		</dependency>


		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-test</artifactId>
			<scope>test</scope>
		</dependency>

		<dependency>
			<groupId>commons-fileupload</groupId>
			<artifactId>commons-fileupload</artifactId>
			<version>1.3.1</version>
		</dependency>

		<!-- https://mvnrepository.com/artifact/com.google.api-client/google-api-client-gson -->
		<dependency>
			<groupId>com.google.api-client</groupId>
			<artifactId>google-api-client-gson</artifactId>
			<version>1.31.2</version>
		</dependency>

		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-web</artifactId>
			<version>5.3.3</version>
		</dependency>

	</dependencies>

	<dependencyManagement>
		<dependencies>
			<dependency>
				<groupId>com.google.cloud</groupId>
				<artifactId>libraries-bom</artifactId>
				<version>16.3.0</version>
				<type>pom</type>
				<scope>import</scope>
			</dependency>
		</dependencies>
	</dependencyManagement>

	<build>
		<plugins>
			<plugin>
				<groupId>org.springframework.boot</groupId>
				<artifactId>spring-boot-maven-plugin</artifactId>
			</plugin>
		</plugins>
	</build>

</project>

2. Just start by implementing the HttpRequestInitializer, Which wraps all the request with google credential.

package com.ashrithgn.google_sotrage_example;

import com.google.api.client.http.HttpRequest;
import com.google.api.client.http.HttpRequestInitializer;
import com.google.auth.http.HttpCredentialsAdapter;
import com.google.auth.oauth2.GoogleCredentials;

import java.io.IOException;
import java.util.Arrays;

public class GoogleHttpRequestInitializer implements HttpRequestInitializer {
    GoogleCredentials credentials;
    HttpCredentialsAdapter adapter;

    @Override
    public void initialize(HttpRequest httpRequest) throws IOException {
        credentials = GoogleCredentials
                .getApplicationDefault()
                .createScoped(Arrays.asList("https://www.googleapis.com/auth/cloud-platform"));
        adapter = new HttpCredentialsAdapter(credentials);

        adapter.initialize(httpRequest);
        httpRequest.setConnectTimeout(60000); // 1 minute connect timeout
        httpRequest.setReadTimeout(60000);

    }

    public GoogleCredentials getCredentials() {
        return credentials;
    }

    public void setCredentials(GoogleCredentials credentials) {
        this.credentials = credentials;
    }

    public HttpCredentialsAdapter getAdapter() {
        return adapter;
    }

    public void setAdapter(HttpCredentialsAdapter adapter) {
        this.adapter = adapter;
    }
}

3. I have Created the Config class which provides the Google Storage client with  connection logic wrapped inside, so this class can be easily used by  auto wiring in spring application.

package com.ashrithgn.google_sotrage_example;


import com.google.api.client.googleapis.javanet.GoogleNetHttpTransport;
import com.google.api.client.http.HttpRequest;
import com.google.api.client.http.HttpRequestInitializer;
import com.google.api.client.json.gson.GsonFactory;
import com.google.api.services.storage.Storage;
import com.google.auth.http.HttpCredentialsAdapter;
import com.google.auth.oauth2.GoogleCredentials;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import java.io.IOException;
import java.security.GeneralSecurityException;
import java.util.Arrays;


@Configuration
public class GoogleStorageCloudConfig {


    @Bean
    Storage configStorageClient() throws GeneralSecurityException, IOException {

        Storage storage = new Storage(GoogleNetHttpTransport.newTrustedTransport(),
                new GsonFactory(), new GoogleHttpRequestInitializer());
        return storage;
    }
}

 4. Writing the adapter class which intern adapts necessary methods and also behave as the translator to the client.

package com.ashrithgn.google_sotrage_example;

import com.google.api.client.http.AbstractInputStreamContent;
import com.google.api.client.http.GenericUrl;
import com.google.api.services.storage.Storage;
import com.google.api.services.storage.model.StorageObject;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;
import org.springframework.web.multipart.MultipartFile;

import java.io.*;

@Component
public class GoogleStorageClientAdapter {
    Storage storage;
    String bucketName;

    public GoogleStorageClientAdapter(@Autowired Storage storage, @Value("${bucket}") String bucketName) {
        this.storage = storage;
        this.bucketName = bucketName;
    }

    public Boolean upload(MultipartFile file, String prefixName) throws IOException {
        StorageObject object = new StorageObject();
        object.setName(file.getOriginalFilename());
        InputStream targetStream = new ByteArrayInputStream(file.getBytes());
        storage.objects().insert(bucketName, object, new AbstractInputStreamContent(file.getOriginalFilename()) {
            @Override
            public long getLength() throws IOException {
                return file.getSize();
            }

            @Override
            public boolean retrySupported() {
                return false;
            }

            @Override
            public InputStream getInputStream() throws IOException {
                return targetStream;
            }
        }).execute();
        return true;
    }


    public StorageObject download(String fileName) throws IOException {
        StorageObject object = storage.objects().get(bucketName, fileName).execute();
        File file = new File("./" + fileName);
        FileOutputStream os = new FileOutputStream(file);

        storage.getRequestFactory()
                .buildGetRequest(new GenericUrl(object.getMediaLink()))
                .execute()
                .download(os);
        object.set("file", file);
        return object;
    }
}

5. Our Rest Controller which upload and downloads the file

package com.ashrithgn.google_sotrage_example;

import com.google.api.services.storage.model.StorageObject;
import com.google.common.io.Files;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.io.ByteArrayResource;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.*;
import org.springframework.web.multipart.MultipartFile;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.File;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;

@RestController
public class Controller {
    @Autowired
    GoogleStorageClientAdapter googleStorageClientAdapter;

    @PostMapping(path = "/upload", consumes = {MediaType.MULTIPART_FORM_DATA_VALUE})
    public Boolean uploadFile(@RequestPart(value = "file", required = true) MultipartFile files)  {
        try {
            return  googleStorageClientAdapter.upload(files, "prefix");
        } catch (IOException e) {
            e.printStackTrace();
        }
        return false;
    }

    @RequestMapping(name = "file-download", path = "download",
            method = RequestMethod.GET)
    public ResponseEntity<ByteArrayResource> fileDownload(HttpServletRequest request,
                                                          @RequestParam(value = "file", required = false) String path,
                                                          HttpServletResponse response
    ) {
        try {
            StorageObject object = googleStorageClientAdapter.download(path);


            byte[] res = Files.toByteArray((File) object.get("file"));
            ByteArrayResource resource = new ByteArrayResource(res);

            return ResponseEntity.ok()
                    .contentLength(res.length)
                    .header("Content-type", "application/octet-stream")
                    .header("Content-disposition", "attachment; filename=\"" + path + "\"").body(resource);
        }catch (IOException e) {
            e.printStackTrace();
            throw new RuntimeException("No such file or directory");
        }
    }
}

And i have also pasted my main class for reference

package com.ashrithgn.google_sotrage_example;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class GoogleSotrageExampleApplication {

	public static void main(String[] args) {
		SpringApplication.run(GoogleSotrageExampleApplication.class, args);
	}

}

Now lets go for cofee break..!!!!!

Useful Reads:

AWS S3 with Spring Boot: Uploading and downloading file to Buckets
So if are reading this Article means your a developer and your familiar withspring boot or AWS S3, just for SEO purpose i will brief both,so you can skipthe introduction part and jump into coding part.As a developer Spring boot hasbecome my vital Framework and preferred choice of man developer, i…
Spring boot: uploading and downloading file Azure Blob Store using rest API
If you have landed on this page means either your working with spring boot andtrying to upload a file to Azure blob storage. so we know spring boot has gainedpopularity for its simple approach to bootstrap an application in very lesstime, Also Azure is gaining a lot of popularity these days as pe…

Follow me on Patreon:

Just Chatter Box is creating Technology & Programming | Patreon
Patreon is a membership platform that makes it easy for artists and creators to get paid. Join over 200,000 creators earning salaries from over 6 million monthly patrons.