Home > Spring > Spring Boot > Spring Boot REST API – Log Incoming HTTP Requests and Responses

Spring Boot REST API – Log Incoming HTTP Requests and Responses

In our article Spring Boot REST API – Log Incoming HTTP Requests we have learned how to log incoming HTTP requests in our Spring Boot REST API application. Here we will learn how we can log both HTTP Requests and Responses in our Spring Boot REST API application.

In this article, we will write our own custom logging solution by creating a Filter, mainly a Servlet Filter. In this logging process, we will generate some kind of unique identifier that will be added to the request and response of the log, which will make our debugging easier. We can extend this approach by pushing our logs to some kind of centralized logging servers such as ELK or others for proper analyzing and debugging. The unique identifier approach will help us to map the incoming request with the corresponding response of that request in the log file.

Lets learn step by step how to log incoming HTTP requests and response in Spring Boot REST API application.

We assume here you already have one Spring Boot REST API application running. So we will directly move to the logging integration part.

  1. Create a logging Filter – ApiRequestResponseLoggingFilter.java

Now we will create ApiRequestResponseLoggingFilter which is nothing but a Servlet Filter. The task of this filter is to intercept all the API requests and responses and log them either in our log file or push them to some kind of centralized log server.

public class ApiRequestResponseLoggingFilter implements Filter {

    private static final Logger LOGGER = LoggerFactory.getLogger(ApiRequestResponseLoggingFilter.class);

    public ApiRequestResponseLoggingFilter() {}

    @Override
    public void init(FilterConfig filterConfig) throws ServletException {}

    @Override
    public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain)
            throws IOException, ServletException {
        try {
            HttpServletRequest httpServletRequest = (HttpServletRequest) request;
            HttpServletResponse httpServletResponse = (HttpServletResponse) response;

            Map<String, String> requestMap = this.getTypesafeRequestMap(httpServletRequest);
            BufferedRequestWrapper bufferedRequest = new BufferedRequestWrapper(httpServletRequest);
            BufferedResponseWrapper bufferedResponse = new BufferedResponseWrapper(httpServletResponse);

            String requestId = UUID.randomUUID().toString(); 
            MDC.put("requestId", requestId);           
            final StringBuilder logRequest = new StringBuilder(requestId).append(" - ").append(" HTTP ")
                    .append(httpServletRequest.getMethod()).append(" \"").append(httpServletRequest.getServletPath())
                    .append("\" ").append(", parameters=").append(requestMap).append(", body=")
                    .append(bufferedRequest.getRequestBody()).append(", remote_address=").append(httpServletRequest.getRemoteAddr());

            LOGGER.info(logRequest.toString());

            try {
                chain.doFilter(bufferedRequest, bufferedResponse);
            } finally {
                final StringBuilder logResponse = new StringBuilder(requestId).append(" - ").append("HTTP RESPONSE ")
                        .append(bufferedResponse.getContent());
                LOGGER.info(logResponse.toString());
                MDC.clear();
            }
        } catch (Throwable a) {
            LOGGER.error(a.getMessage());
        }
    }

    private Map<String, String> getTypesafeRequestMap(HttpServletRequest request) {
        Map<String, String> typesafeRequestMap = new HashMap<String, String>();
        Enumeration<?> requestParamNames = request.getParameterNames();
        while (requestParamNames.hasMoreElements()) {
            String requestParamName = (String) requestParamNames.nextElement();
            String requestParamValue;
            if (requestParamName.equalsIgnoreCase("password")) {
                requestParamValue = "********";
            } else {
                requestParamValue = request.getParameter(requestParamName);
            }
            typesafeRequestMap.put(requestParamName, requestParamValue);
        }
        return typesafeRequestMap;
    }

    @Override
    public void destroy() {
    }

    private static final class BufferedRequestWrapper extends HttpServletRequestWrapper {

        private ByteArrayInputStream bais = null;
        private ByteArrayOutputStream baos = null;
        private BufferedServletInputStream bsis = null;
        private byte[] buffer = null;

        public BufferedRequestWrapper(HttpServletRequest req) throws IOException {
            super(req);
            // Read InputStream and store its content in a buffer.
            InputStream is = req.getInputStream();
            this.baos = new ByteArrayOutputStream();
            byte buf[] = new byte[1024];
            int read;
            while ((read = is.read(buf)) > 0) {
                this.baos.write(buf, 0, read);
            }
            this.buffer = this.baos.toByteArray();
        }

        @Override
        public ServletInputStream getInputStream() {
            this.bais = new ByteArrayInputStream(this.buffer);
            this.bsis = new BufferedServletInputStream(this.bais);
            return this.bsis;
        }

        String getRequestBody() throws IOException {
            BufferedReader reader = new BufferedReader(new InputStreamReader(this.getInputStream()));
            String line = null;
            StringBuilder inputBuffer = new StringBuilder();
            do {
                line = reader.readLine();
                if (null != line) {
                    inputBuffer.append(line.trim());
                }
            } while (line != null);
            reader.close();
            return inputBuffer.toString().trim();
        }
    }

    private static final class BufferedServletInputStream extends ServletInputStream {

        private ByteArrayInputStream bais;

        public BufferedServletInputStream(ByteArrayInputStream bais) {
            this.bais = bais;
        }

        @Override
        public int available() {
            return this.bais.available();
        }

        @Override
        public int read() {
            return this.bais.read();
        }

        @Override
        public int read(byte[] buf, int off, int len) {
            return this.bais.read(buf, off, len);
        }

        @Override
        public boolean isFinished() {
            return false;
        }

        @Override
        public boolean isReady() {
            return true;
        }

        @Override
        public void setReadListener(ReadListener readListener) {

        }
    }

    public class TeeServletOutputStream extends ServletOutputStream {

        private final TeeOutputStream targetStream;

        public TeeServletOutputStream(OutputStream one, OutputStream two) {
            targetStream = new TeeOutputStream(one, two);
        }

        @Override
        public void write(int arg0) throws IOException {
            this.targetStream.write(arg0);
        }

        public void flush() throws IOException {
            super.flush();
            this.targetStream.flush();
        }

        public void close() throws IOException {
            super.close();
            this.targetStream.close();
        }

        @Override
        public boolean isReady() {
            return false;
        }

        @Override
        public void setWriteListener(WriteListener writeListener) {

        }
    }

    public class BufferedResponseWrapper implements HttpServletResponse {

        HttpServletResponse original;
        TeeServletOutputStream tee;
        ByteArrayOutputStream bos;

        public BufferedResponseWrapper(HttpServletResponse response) {
            original = response;
        }

        public String getContent() {
            return bos.toString();
        }

        public PrintWriter getWriter() throws IOException {
            return original.getWriter();
        }

        public ServletOutputStream getOutputStream() throws IOException {
            if (tee == null) {
                bos = new ByteArrayOutputStream();
                tee = new TeeServletOutputStream(original.getOutputStream(), bos);
            }
            return tee;

        }

        @Override
        public String getCharacterEncoding() {
            return original.getCharacterEncoding();
        }

        @Override
        public String getContentType() {
            return original.getContentType();
        }

        @Override
        public void setCharacterEncoding(String charset) {
            original.setCharacterEncoding(charset);
        }

        @Override
        public void setContentLength(int len) {
            original.setContentLength(len);
        }

        @Override
        public void setContentLengthLong(long l) {
            original.setContentLengthLong(l);
        }

        @Override
        public void setContentType(String type) {
            original.setContentType(type);
        }

        @Override
        public void setBufferSize(int size) {
            original.setBufferSize(size);
        }

        @Override
        public int getBufferSize() {
            return original.getBufferSize();
        }

        @Override
        public void flushBuffer() throws IOException {
            tee.flush();
        }

        @Override
        public void resetBuffer() {
            original.resetBuffer();
        }

        @Override
        public boolean isCommitted() {
            return original.isCommitted();
        }

        @Override
        public void reset() {
            original.reset();
        }

        @Override
        public void setLocale(Locale loc) {
            original.setLocale(loc);
        }

        @Override
        public Locale getLocale() {
            return original.getLocale();
        }

        @Override
        public void addCookie(Cookie cookie) {
            original.addCookie(cookie);
        }

        @Override
        public boolean containsHeader(String name) {
            return original.containsHeader(name);
        }

        @Override
        public String encodeURL(String url) {
            return original.encodeURL(url);
        }

        @Override
        public String encodeRedirectURL(String url) {
            return original.encodeRedirectURL(url);
        }

        @SuppressWarnings("deprecation")
        @Override
        public String encodeUrl(String url) {
            return original.encodeUrl(url);
        }

        @SuppressWarnings("deprecation")
        @Override
        public String encodeRedirectUrl(String url) {
            return original.encodeRedirectUrl(url);
        }

        @Override
        public void sendError(int sc, String msg) throws IOException {
            original.sendError(sc, msg);
        }

        @Override
        public void sendError(int sc) throws IOException {
            original.sendError(sc);
        }

        @Override
        public void sendRedirect(String location) throws IOException {
            original.sendRedirect(location);
        }

        @Override
        public void setDateHeader(String name, long date) {
            original.setDateHeader(name, date);
        }

        @Override
        public void addDateHeader(String name, long date) {
            original.addDateHeader(name, date);
        }

        @Override
        public void setHeader(String name, String value) {
            original.setHeader(name, value);
        }

        @Override
        public void addHeader(String name, String value) {
            original.addHeader(name, value);
        }

        @Override
        public void setIntHeader(String name, int value) {
            original.setIntHeader(name, value);
        }

        @Override
        public void addIntHeader(String name, int value) {
            original.addIntHeader(name, value);
        }

        @Override
        public void setStatus(int sc) {
            original.setStatus(sc);
        }

        @SuppressWarnings("deprecation")
        @Override
        public void setStatus(int sc, String sm) {
            original.setStatus(sc, sm);
        }

        @Override
        public String getHeader(String arg0) {
            return original.getHeader(arg0);
        }

        @Override
        public Collection<String> getHeaderNames() {
            return original.getHeaderNames();
        }

        @Override
        public Collection<String> getHeaders(String arg0) {
            return original.getHeaders(arg0);
        }

        @Override
        public int getStatus() {
            return original.getStatus();
        }
    }
}

2. Now lets configure the Logging Filter with the configurable properties from the application.properties file.

@Component
@ConditionalOnExpression("${app.api.logging.enabled:true}")
public class MusicApiRequestResponseLoggingFilterConfig {

    @Value("${app.api.logging.url-patterns:*}")
    private String[] urlPatterns;

    @Bean
    public FilterRegistrationBean<MusicApiRequestResponseLoggingFilter> loggingFilter() {
        FilterRegistrationBean<MusicApiRequestResponseLoggingFilter> registrationBean = new FilterRegistrationBean<>();
        registrationBean.setFilter(new MusicApiRequestResponseLoggingFilter(requestIdParamName, requestIdMDCParamName));
        registrationBean.addUrlPatterns(urlPatterns);
        return registrationBean;
    }
}

Here we have used two configurable properties first is app.api.logging.enabled, which will tell the application to log request and response or not. if it is configured with the value true, logging will be enabled else disabled. By default, if it is not mentioned in the application.properties file it will consider it as enabled.

Second is app.api.logging.url-patterns, this provides the comma separated API URL patterns, whose request and responses are required to be logged. If not provided we have written ‘*‘, meaning all the API needs to be logged.

Sample values of these two in application.properties file are –

app.api.logging.enabled=true
app.api.logging.url-patterns=/product/*,/purchase/*

Here we have enabled the logging and have configured API’s with URL pattern /product/ and /purchase/ are only required for logging.