Making the case for Jython (again)

Writing PDF reports with Django integrating Jython and DynamicReports

Posted by Agustín Bartó 2 years, 9 months ago Comments

Motivation

A few days ago Jython 2.7.0 was released and in order to celebrate we want to revisit something that we introduced in a previous blogpost.

Back then we argued that Jython had a place in a Python eco-system because it allowed the integration of Java libraries that cover functionality that isn’t readily available in the Python world. Sadly, the examples that we presented had very narrow use cases.

This time we’ll show you something that you might end up using on your own projects.

The Problem

A colleague asked us for help in tackling a problem in a Django project. Their reporting solution was based on Relatorio and they were concerned with the library as it seemed that it was abandoned. On top of this, the templates were stored as binary files which make the version tracking and bug-fixing a real chore.

As we mentioned in the previous article, some of us have a lot of experience with Java, and most of us had to tackle reporting more than once. Luckily, there are quite a few mature reporting solutions in Java like JasperReports or DynamicReports.

Given that the existing solution was already dependent on external tools like LibreOffice, trying to integrate an alternative mature non-python library might be worth the try.

The Solution

We cannot show you the code that needed the new reporting solution, so we’ll use a Django project presented in an older blogpost. You don’t have to worry about the particulars or the project, since we’re only going to be generating a simple tabular report of a particular entity. Everything is available on GitHub.

The proposed solution was to use DynamicReports, but generating the report structure and content from Python instead of Java. Each report is contained in a module that takes the data from the command line (in Base64 encoded JSON) and returns the a Base64 encoded PDF report on the standard output. We know the solution is far from ideal, but it’ll give you a starting point for a more adequate architecture.

The first problem to tackle is packaging Jython and DynamicReports (and all their dependencies) in a way that makes the deployment and execution as simple as possible. This can be done easily in Java using Maven. For those unfamiliar with Maven, it is a tool to manage the project workflow, from building to packaging to documentation and much, much more. A Maven project is defined using a project object model (POM) file. On this file you can declare the project dependencies, define how it is built, how to execute tests, etc. A little bit like you would do with fabric and pip.

You can see the full version of pom.xml file on GitHub, but for now, let us concentrate on the dependencies:

<dependencies>
        <dependency>
                <groupId>net.sourceforge.dynamicreports</groupId>
                <artifactId>dynamicreports-core</artifactId>
                <version>4.0.0</version>
        </dependency>
        <dependency>
                <groupId>org.python</groupId>
                <artifactId>jython-standalone</artifactId>
                <version>2.7.0</version>
        </dependency>
</dependencies>

Our project only depends on Jython and DynamicReports, but these packages have dependencies of their own. Deploying Java packages can be problematic (specially when multiple conflicting jars are involved), and we want to keep our deployment simple, so we’ll make use of Maven’s Shade Plugin. This plugin allows us to build a single JAR file containing all the dependencies. Here’s the configuration for the plugin:

<plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-shade-plugin</artifactId>
        <version>2.3</version>
        <executions>
                <execution>
                        <phase>package</phase>
                        <goals>
                                <goal>shade</goal>
                        </goals>
                        <configuration>
    <createDependencyReducedPom>false</createDependencyReducedPom>
                                <transformers>
                                        <transformer
                                                implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                                                <manifestEntries>
                                                        <Main-Class>org.python.util.jython</Main-Class>
                                                </manifestEntries>
                                        </transformer>
                                </transformers>
                                <filters>
                                        <filter>
                                                <artifact>net.sourceforge.dynamicreports:dynamicreports-core</artifact>
                                                <excludes>
                                                        <exclude>jasperreports_extension.properties</exclude>
                                                </excludes>
                                        </filter>
                                </filters>
                        </configuration>
                </execution>
        </executions>
</plugin>

What we’re doing here is telling Maven to generate a single JAR file, using org.python.util.jython (the Jython shell) as the Main class. We’re also filtering (i.e. not including in the final package) a file that causes problems when executing the JAR directly.

In order to build the package, you need to install Maven. In a ubuntu system this can be accomplished with the following command:

$ sudo apt-get install maven

Once the installation finishes, you can build the package invoking the following command from the directory that contains the pom.xml file:

$ mvn clean package

If you haven’t worked with maven before, and depending on your Internet connection it might take a while for the command to complete as it has to download all the required Maven components and the project’s dependencies. If everything went well there should be a file named reports-0.0.1.jar on the “target/” directory.

It might surprise you that no Java code is needed for this to work. The rest of the component is all written in Python. Granted, we make use of existing Java classes but once you’re familiar with reading javadocs, you’ll be able to leverage the full power of the DynamicReports library (or any other Java library).

We’re now going to write a Python module to generate PDF reports of Incidents using the DynamicReports API.

We could make use of PostgreSQL’s JDBC drivers to access the database directly, but we rather keep this as simple as possible so the script will take a Base64 encoded JSON representation of the Incident list. Using this, we build a DataSource object that’s used to describe and build the actual report. Once the report is built, we print the Base64 encoded result to the standard output.

import sys

from base64 import (b64encode, b64decode)
from java.io import ByteArrayOutputStream
from json import loads

from net.sf.dynamicreports.report.builder.DynamicReports import (
    cmp as dr_cmp, col as dr_col, report as dr_report, type as dr_type
)
from net.sf.dynamicreports.report.datasource import DRDataSource


def get_report_data(report_data):
    return loads(b64decode(report_data).decode("utf-8"))


def create_report(data_source):
    output_stream = ByteArrayOutputStream()
    dr_report() \
        .columns(
            dr_col.column('Pk', 'pk', dr_type.integerType()),
            dr_col.column('Name', 'name', dr_type.stringType()),
            dr_col.column('Description', 'description', dr_type.stringType()),
            dr_col.column('Severity', 'severity', dr_type.stringType()),
            dr_col.column('Closed', 'closed', dr_type.booleanType()),
            dr_col.column('Location', 'location', dr_type.stringType()),
            dr_col.column('Created', 'created', dr_type.stringType())
        ) \
        .title(dr_cmp.text('Incidents')) \
        .setDataSource(data_source) \
        .toPdf(output_stream)

    return output_stream.toByteArray()


def create_data_source(data):
    dr_data_source = DRDataSource(
        ['pk', 'name', 'description', 'severity', 'closed', 'location', 'created']
    )

    for incident in data:
        dr_data_source.add(
            incident['pk'],
            incident['fields']['name'],
            incident['fields']['description'],
            incident['fields']['severity'],
            incident['fields']['closed'],
            incident['fields']['location'],
            incident['fields']['created']
        )

    return dr_data_source

if __name__ == '__main__':
    data_source = create_data_source(get_report_data(sys.argv[1]))
    report_pdf = create_report(data_source)

    print b64encode(report_pdf)

The generated report is rudimentary, but after reading the documentation you’ll be able to write incredibly rich reports. This set-up solves the problem the report design version tracking, as it is just a plain Python module.

In order to use this module, we created a new Django view that invokes java supplying the report data, and sends the results back to the client:

@login_required()
def incident_export_report(request):
    response = HttpResponse(content_type='application/pdf')
    response['Content-Disposition'] = 'attachment; filename="incidents-{}.pdf"'.format(datetime.now())

    incidents = Incident.objects.all().order_by('-created')
    report_data = b64encode(serialize('json', incidents).encode('utf-8'))

    reports_output = check_output(
        ['java', '-jar', 'tracker/reports/target/reports-0.0.1.jar', 'tracker/reports/incidents.py', report_data],
        stderr=None, stdin=None
    )

    response.write(b64decode(reports_output.strip()))

    return response

Conclusions

As you saw, Jython allowed us to integrate a fairly mature reporting library without much effort. The interaction between Python and Java through the command line is not very elegant, but it can be easily replaced by other solutions.

The presented approach can be used with any other Java library just by changing the dependency declaration on the POM file.

We think that we’ve made a very good case for the usage of Jython in a Python eco-system and that you shouldn’t shy away from the incredibly rich Java libraries that are available.

As usual, any comments or suggestions are welcomed. We’re interested to know if you had to face a similar problem where a Java library might replace a Python module that’s not ideal for your needs.


Previous / Next posts


Comments