Posts in Category: Programming

This category contains articles related to programming

AEM integration with webp

A question from my colleagues in regards to AEM’s support for webp started a research path. Since, I took more than 1 year pause from AEM, I started looking around for the possibility of serving also webp images, for all the images already in DAM.

As a first step I started ask the community on stackoverflow and Adobe Forum. The answer received was to use a webp java library that provides support for reading and writing webp images .

Note: You will notice I have usedcom.github.nintha library. At the end of the article I will explain the reasoning for it and how I actually ditched it to work.

The steps for resolving this were:

  1. To add the 3rd party jar to AEM
  2. Create a simple servlet that receives as queryparam the image path as and returns the webp image.

Adding the 3rd party jar

In AEM one can install via in Apache Felix only bundled jar, for the simple jars, one can embed them using the maven-bundle-plugin in tree steps:

  1. Add the library jar to your application pom.xml (the main one)
<dependency>
<groupId>com.github.nintha</groupId>
<artifactId>webp-imageio-core</artifactId>
<version>0.0.1</version>
</dependency>

2. Modify the pom.xml from the core app to add the dependency and embed the jar. The core/pom.xml will have something like:

<build>
<plugins>
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-scr-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
<extensions>true</extensions>
<configuration>
<instructions>
<Import-Package>javax.inject;version=0.0.0,*</Import-Package>
<Export-Package>andra.core.*</Export-Package>
<Sling-Model-Packages>
andra.core
</Sling-Model-Packages>
<Embed-Dependency>webp-imageio-core</Embed-Dependency>
<Embed-Transitive>true</Embed-Transitive>
</instructions>
</configuration>
</plugin>
</plugins>
</build>
....
<dependency>
<groupId>com.github.nintha</groupId>
<artifactId>webp-imageio-core</artifactId>
</dependency>

Things to mention: the Embed-Dependency tag needs to be populated with the artifactId. The <Embed-Transitive> tag specifies to embed also the dependencies of the embedded jar. (eg. the webp-imageio-core has as dependency org.scijava – native-lib-loader, which will be embedded as well)

In order to test that your 3rd party was successfully embedded, one can take a look at the bundle information:

Verify that:
– Bundle Classpath includes the 3rd party jar
– The manifest header contains the embedded dependencies, artifacts and transitive set as expected

Create the Servlet to transform to webp

For the purposes of this POC, it will be only a basic Servlet, it can be enhanced, of course to work with image renditions, or to work receive the path as a suffix – which would be the right way as we want the images to be cached.

package andra.core.servlets;


import com.day.cq.dam.api.Asset;
import com.luciad.imageio.webp.WebPWriteParam;
import org.apache.felix.scr.annotations.Properties;
import org.apache.felix.scr.annotations.Property;
import org.apache.felix.scr.annotations.sling.SlingServlet;
import org.apache.sling.api.SlingHttpServletRequest;
import org.apache.sling.api.SlingHttpServletResponse;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.api.resource.ResourceResolver;
import org.apache.sling.api.servlets.HttpConstants;
import org.apache.sling.api.servlets.SlingSafeMethodsServlet;
import org.osgi.framework.Constants;

import javax.imageio.IIOImage;
import javax.imageio.ImageIO;
import javax.imageio.ImageWriteParam;
import javax.imageio.ImageWriter;
import javax.imageio.stream.ImageOutputStream;
import javax.imageio.stream.MemoryCacheImageOutputStream;
import javax.servlet.ServletException;
import java.awt.image.BufferedImage;
import java.io.IOException;

@SlingServlet(paths = "/bin/webpTransformer",
methods = HttpConstants.METHOD_GET
)
@Properties({
@Property(name = Constants.SERVICE_DESCRIPTION, value = "WebP Transformer Servlet")
})
public class WepServlet extends SlingSafeMethodsServlet {


public static final String IMAGE_WEBP = "image/webp";


@Override
protected void doGet(SlingHttpServletRequest request, SlingHttpServletResponse response) throws ServletException, IOException {
response.setContentType(IMAGE_WEBP);

BufferedImage image = getImage(request);

// Obtain a WebP ImageWriter instance
ImageWriter writer = ImageIO.getImageWritersByMIMEType(IMAGE_WEBP).next();

// Configure encoding parameters
WebPWriteParam writeParam = new WebPWriteParam(writer.getLocale());
writeParam.setCompressionMode(ImageWriteParam.MODE_EXPLICIT);
writeParam.setCompressionType("Lossless");
ImageOutputStream outputStream = new MemoryCacheImageOutputStream(
response.getOutputStream());

// Configure the output on the ImageWriter
writer.setOutput(outputStream);

// Encode
writer.write(null, new IIOImage(image, null, null), writeParam);
outputStream.flush();
outputStream.close();


}

private BufferedImage getImage(SlingHttpServletRequest request) throws IOException {
ResourceResolver resourceResolver = request.getResourceResolver();
String path = request.getParameter("path");
Resource res = resourceResolver.getResource(path);
Asset asset = res.adaptTo(Asset.class);
return ImageIO.read(asset.getOriginal().getStream());
}

}

Why and how nintha ?

A question I would ask is why use nintha over luciad, as in the end nintha is just a luciad wrapper. Of course, I always try to use the standard lib, where possible, however when embedding luciad compiled on my machine I would get the following error:

Could not initialize class com.luciad.imageio.webp.WebPEncoderOptions
Cannot serve request to /bin/webpTransformer in andra.core.servlets.WepServlet

Exception:
java.lang.NoClassDefFoundError: Could not initialize class com.luciad.imageio.webp.WebPEncoderOptions
at com.luciad.imageio.webp.WebPWriteParam.(WebPWriteParam.java:30)
at com.luciad.imageio.webp.WebPWriter.getDefaultWriteParam(WebPWriter.java:38)

After a couple of days, I decided to give nintha a go. However, it did not go as smooth as I wanted, as know I would receive:

org.scijava.nativelib.NativeLibraryUtil Problem with library
java.lang.UnsatisfiedLinkError:

Meaning, that the native library included in nintha was not compatible with the OS the app was running. So, I had to add compile the native app on my local machine, add it manually to the nintha library and recomile. This is not a solution I like as it would be a downside when migration to another server – the library needs to encapsulate the native app for the specific environment.

Javaist learning Python – Part 2 – conditions

I will write about what something I found funny – the conditions

  1. Python has a elif statement. This makes the code easier to read. Python you have a +1 from me for this
  2. Ok… now we go to the part where we find that one does not need to check for a boolean condition. Not only it does not need to check for this, it is advised NOT to check against False, True or None (a singleton that is somehow similar to the null in Java). Everything except the next statements are considered True(This reminded me of highschool and C++):
    • None
    • False
    • zero – for numeric types
    • empty dictionaries, sequences
    • 0 returned when calling __length__
    • False returned when __nonzero__ is called
  3. Now, an even stranger thing: one can have an else after a for… say whaat? yep… One can face something like
    for loop:
        if condition:
            ….
            break
    else:
        statement
    So, what is this all about? The statement is executed when after the program finished the loop… Every time, with one exception: when a break was executed inside the loop. I still have mixed feelings about this, but probably after using more of it, it will come natural.

This was all from my Python adventure till this moment.

Javaist learning Python – Part 1

I always enjoyed stuff that had to do with programming languages and learning how to set your mind to new things/paradigms. One of my favourite courses in the University was Logic and Functional Programming – learning Prolog and LISP (Let’s Insert Some Parenthesis :D).

I played a little bit with Python for my Mater’s Thesis (Static vs Dynamic Typed Programming Languages) some years ago and I always wanted to go a little bit more in depth with it. So, now I considered it to be a great time to restart playing with Python.

The syntax I consider it to be pretty straightforward and there is quite a good documentation on the internet. But, somehow I felt I was missing the Java API docs format. For me it is more convenient to search for something and see what else I can do with that object. Also, the font size seems to be kind of unfriendly – small and a little bit cramped(I know one can resolve this with zoom on the page). I will probably get used to the documentation, but in the meantime I will crave for Java documentation.

Even though, I knew it is not a pure object oriented programming language, is it still funny to white something “outside an object”, but this did not bother me at all, on the contrary, I found it really easy to get started writing code.

Another thing I really like the command line interpretor, that can be run from the command line. Having this is really helpful for the people that have their first contacts with Python. One can actually test something before adding it to a program.

That’s all for now, but I will keep posting further impressions :).

Collections quick review

There are times in our developer lives when we just settle with a range of implementations without thinking of the reasons behind it, or without having any valid argument in order to use one implementation over another.

Some time ago I was having a discussion with a friend [which is a big data developer] about the importance of choosing the right collection. He gave an example on how using the wrong implementation a job took over 3 hours, after changing the implementation of the collection used the same job needed 5 minutes to complete.

Below can be seen a table with basic information about different implementations of collections

Implementation Ordered Sorted Syncronized Nulls Notes
HashMap One null key andno null values
Hashtable* No null keys and no null values
TreeMap Accepts nulls as keys and value
LinkedHashMap Accepts nulls as keys and value Faster iterations than HashMap but slower inserts and remove
HashSet Accepts null
TreeSet Accepts null
LinkedHashSet Accepts null
ArrayList Accepts null Fast iterations
Vector Accepts null
LinkedList Accepts null Fast iterations and deletion
PriorityQueue Accepts null

* There is not a typo: Hashtable is the special kid that does not have camel case

PS: This is just a quick review for more about how each works, I suggest visiting the API.

So what’s that difference again? Abstract class vs Interface

Everything was nice and pretty clear before beautiful Java 8 was born.  One could define:

(1) Abstract class as being a class declared abstract (usually, but not necessary, contains at least one abstract method).

(2) Abstract method as being one method that is declared, but for which no implementation was provided.

(3) Interface as a 100% abstract class.

Even though, not all the differences are explained in these statements, these would have been acceptable definitions.

And then Java 8 was brought into the world, and the new feature (considered by some the best new enhancement) : default and static methods for interfaces. Non-abstract methods defined directly in the interface. This implies that (3) became obsolete and the magic question arise:

What is the difference between an interface and abstract class? 

Abstract class Interface
Default methods any method that is not abstract can be considered defaultit is not marked with default modifier marked withdefault modifier
Static methods static modifier required see below examples static modifier required
Non-Constants variables can be defined all the variables are implicitly public static final
A Java class can extend just one class (abstract or concrete) implement multiple interfaces
A Java interface cannot extend/implement an abstract class can extend multiple interfaces
The first concrete class extending the abstract class must implement all abstract methods implementing the interface must implement all abstract methods
Implicit modifiers for variables No implicit modifiers public static final
Implicit modifier for methods No implicit modifiers the non-static & non-default methods are implicitly public abstract
When to use … ? when the code is shared amongst related classes when the code is shared among unrelated classes

So, let’s revisit the definition of an interface. I would like to go for this one:

Interfaces are abstract data types that provide a contract between its users and its providers.

Here you have a nice example to start discovering the differences between abstract classes and interfaces!

import java.io.*;
import static java.lang.System.out;
class Main {
 public static void main (String[] args) throws Exception { //Mother mother = new Mother(); DOES NOT COMPILE
 //AnInterface interface = new AnInterface(); DOES NOT COMPILE
 Mother child1 = new Child();
 AnInterface child2 = new Child();
 Child child3 = new Child();
 
 //calling static fields
 out.println(child1.ABSTRACT_CLASS_NAME);
 out.println(Mother.ABSTRACT_CLASS_NAME);
 out.println(child3.ABSTRACT_CLASS_NAME);
 out.println(Child.ABSTRACT_CLASS_NAME);

 out.println(child2.INTERFACE_NAME);
 out.println(AnInterface.INTERFACE_NAME);
 out.println(child3.INTERFACE_NAME);
 out.println(Child.INTERFACE_NAME);

 //caling static methods
 child1.staticMotherMethod();
 Mother.staticMotherMethod();
 child3.staticMotherMethod();
 Child.staticMotherMethod();
 //calling default methods

 //child2.staticInterfaceMethod(); DOES NOT COMPILE
 AnInterface.staticInterfaceMethod();
 //child3.staticInterfaceMethod(); DOES NOT COMPILE
 //Child.staticInterfaceMethod(); DOES NOT COMPILE
 
 //calling default methods
 child1.defaultMotherMethod();
 child1.defaultMotherMethod1();
 child3.defaultMotherMethod();
 child3.defaultMotherMethod1(); 

 child2.defaultInterfaceMethod();
 child2.defaultInterfaceMethod1();
 child3.defaultInterfaceMethod(); 
 child3.defaultInterfaceMethod1();

 }
}

abstract class Mother {

 public static final String ABSTRACT_CLASS_NAME = "Mum name";

 public static void staticMotherMethod() { //the access modifier is not implied
 out.println("static Mother Method");
 }

 public void defaultMotherMethod() { //the access modifier is not implied
 out.println("Mother default");
 }

 public void defaultMotherMethod1() {
 out.println("Mother default 1"); //the access modifier id not implied
 }

 public abstract void abstractMotherMethod() ;
}

interface AnInterface {

 String INTERFACE_NAME = "Interface name"; //implicit public static final

 default void defaultInterfaceMethod() { //implicit public
 out.println("default interface method");
 }

 default void defaultInterfaceMethod1() {
 out.println("default interface 1");
 }

 static void staticInterfaceMethod() { //implicit public
 out.println("static inteface method");
 }

 void abstractInterfaceMethod(); //implicit public abstract
}

class Child extends Mother implements AnInterface {
 
 public void abstractInterfaceMethod() {
 out.println("Child abstractInterfaceMethod");
 }

 public void abstractMotherMethod() {
 out.println("Child abstractMumMethod");
 }

 public void defaultMotherMethod1() {
 out.println("Child default method");
 }

 public void defaultInterfaceMethod() {
 out.println("Child interface method");
 }

} 

Hope you had fun! 🙂 Till the next post hope you enjoy Java!

Everybody likes promotions

…of course, I talk about the Java numeric promotions. Probably everyone knows what she/he should do to get promoted, but what about the Java numeric promotions?

A. What does promotion mean?

The numeric promotion represents the automatically made conversion from a smaller numeric type to a wider numeric type when a binary aritmetic operation is made. [So it seems that this does not require hard work for some :D]

Just a small recap about the types and their length can be seen in the below table:

Type length in bytes length in bits
byte 1  1 * 8 = ?
short 2 please guess
int 4 hard one
long 8  ∞ * ∞ (reversed)
float 4 32
double 8 be my guest
char 2 16

B. What is required to get a promotion?

Just operands of the same type can be used for a binary operation. In order to achieve that there are some clear rules about how promotions occur:

1. When applying the operator for operands of different types: the smaller numeric type is promoted to the larger number type

2. When applying the operator for an integer value and a floationg point : the integer value is promoted to the floating point value.

3. Byte, short, char are automatically promoted to int when using binary operator.

4. The result of the operation has the type of the operands. [Mind the fact at the time the operator is applied the operands have the same type]

C. Tiny games with promotions

1. Let’s apply the above mentioned rules for the follwing example:

byte byteA = 1;
byte byteB = 2;
//byte byteSum = byteA + byteB;
//short shortSum = byteA + byteB;
int intSum = byteA + byteB;

Applying rule number 3 it is clear that the commented out lines of code result in a not very elegant compilation error:

 error: possible loss of precision
 byte byteSum = byteA + byteB;
 required: byte
 found: int

2.  Apply the rules and say the type .

int intA = 1;
float floatB = 2.0f;
byte byteC = 3;
____ sum = intA % byteC + floatB;

Yep, first rule number 1 is applied for byteC and the result of intA % byteC would be an integer, then this result is promoted to float and the returned value is float.

3. Let’s try another game. Take the following code:

byte byteA = 1;
byte byteB = 2;
byteB += byteA;

What happens? If you said “Compilation failure”, please, think again. As I said the promotion is applied only for binary arithmetic operations and += is actually considered a compound assignment operation, but that is another story 🙂

People tend do enjoy promotions ;). I bet you get a promotion every day, so enjoy today!

And I fell in love with the date…Java 8 Date

Love is in the air year after year in the period when winter and spring fight for power.

In the week that Java8 turns one I am going to speak about the date I fell in love with…Java 8 Date.

Even though I have never considered the previous implementation of Dates as being a pain (just a little bit verbose :p) I am really appreciative and using it made me fall instanlty in love with the new Date.

What do I really like about Date? You may find listed below some(not all) reasons:

  • The possibility to use classes specify/use exactly what you need:

Why should I be forced to store the specific date (usually 1st January 1970) when I just need the hours?  was a question that arose probably in many situations. [I would really want to know how many GB of data occupies the 1st January 1970]

Java 8 resolved the issue with adding specific classes LocalDateTime, LocalDate and LocalTime classes for the “un-timezoned” feature and for each of these provides the zoned class, which stores also the timezone for each of them.

Also, the precision one needs to use can be specified

  • The easiness to get a reference to a new instance of a Date.

Seems like a dream not to use a Calendar class to get a new instance of a Date class:

LocalDateTime thisMoment = LocalDateTime.now();

LocalDate java8LauchDate = LocalDate.of(18, Month.MARCH, 2014)

LocalDate forTheNostalgics = LocalDate.ofEpochDay(1);

  • The beauty of the Period and Duration

Until this moment we specified precise moment in time, but sometimes we need to use an interval rather than specific points in time.

Period oneYearTwoMonthsThreeDays = Period.of(1,2,3);

and Period’s little brother (used for intervals within a day):

Duration snoozeDuration = Duration.ofMinutes(10);

  • One can do maths![How can one not love that]

The addition/substraction of specific periods is just a bliss:

LocalDateTime now = LocalDateTime.now().plusWeeks(1).plusDays(3).minusDays(7);

  •  The usage of the “normal” number for the month.

Just imagine in a previous Java version my birthday would have been declared like. Ok, I am  a developer and used with 0-index, but I could not just set my brain to use 5 0 for my birthday[Now that you know when my birthday is, you can send me birthday cards or presents and I do not mind receiving them before of after my bithday…so it’s still not too late 🙂 ]

Happy birthday, Java8!

And for you: hope you all have dates as good as the Java 8 ones!

Mule ESB create MongoDB 2dsphere Index

Mule ESB 3.5 works like a bliss with MongoDB 2.4.9,  providing an easy way to configure the connection. A template for MongoDB configuration looks like:

<mongo:config doc:name=”MongoDb”
name=”MongoDatabase”
database=”%databaseName%”
username=”%databaseUsername%”
password=”%databasePassword%”
host=”%databaseHost%”
port=”%databasePort%”>

The MongoDB connector provided, gives the user the possibility to easily configure almost any database operation.

But what should be done when the operation we want to perform is not quite straightforward? An example to this is creating a 2dsphere index. Even though, the MongoDB connector provides “Create Index” operation, one can configure just the order of the index, but not its type.

An approach for these situations without the necessity to  write custom code is to use the “Execute command” operation provided. This operation is a representation of db.runCommand.

So, comming back to the 2dsphere index, the creation using db.runCommand would normally look like:

db.runCommand( { eval :function() {db.collectionName.ensureIndex( {geolocation : ‘2dsphere’} ) } } )

In case we want to create the index using Mule the following connector configuration is to be done:

MongoDB mule

or if you prefer XML version, the previous photo is translated into:

<mongo:execute-command config-ref=”MongoDatabase” commandName=”eval” commandValue=”function(){db.collectionName.ensureIndex({geolocation : ‘2dsphere’})}” doc:name=”Mongo DB”/>

The usage of the “Execute command” operation seems to be straightforward, but at the moment this article was written the documentation for mule:execute-command was lacking a proper example.

Does an epidemics affects us on a psychological level?

On 19-11-2014 the BigData Community from Cluj-Napoca held the  4th BigData/DataScience Cluj-Napoca meetup and 1st Contest with prizes offered by TrustYou. The contest’s theme was : “Take  GDELT datasets and do crazy things with them!”. This contest gave me the chance to have my first encounter with BigData and Data Analysis [ iei 🙂 ].

 

10625063_10152917940029271_253315311961567841_n

 

Choosing the theme

Observing that the illness or potential risk of our acquaintances sadden us or make us be more aware of that problem, I wanted to see how this is manifested on a larger scale.

Also, a target for this study was to see if in the areas that risk an epidemics psychological aid is needed.

The criteria to choose the country for my study were:

  • close enough to the place Ebola appeared – Papa New Guinea,  in order to be at a potential risk,
  • far enough not to be a 100% target.
  • affected by Ebola in 2014
  • declared Ebola free.

The above criteria made me pick Nigeria for the study.

The study

First question I asked myself was how the psychological impact  could be determined. I took for the impact measurement the  number of suicides.

Using GDELT Analysis Service I simulated a timed heat  map for every week from 30.12.2013 to 20.10.2014 for GDELT GKG Nigeria and suicidal mentions.

The heat map is a graphic representation with the values represented as colours. For GDELT as the number of mentions for the searched  words increases, the colour gets warmer (few mentions are represented as blue, while a large number of mentions is represented by red)

A preview for the study  can be seen in the next gif.

multiple

It can be seen that at the beginning of the year there have not been many suicidal mentions in the media, for example the firth two weeks of February have no such mentions. (Maybe everyone was happy before Valentine’s day 😀 ). After 23th March, when WHO (Word Health Organization) declares the outbreak of Ebola the number of suicidal mentions in Nigeria increases in a visible way.

On 20th July Ebola is introduces in Nigeria. The person who introduced Ebola in Nigeria negated any connections with Ebola, that is the reason he was treated for Meningitis. A week after, he dies from Ebola as well as his taxi driver. Part of his medical stuff gets infected with Ebola and for some the disease is fatal. During the summer further Ebola cases appear. The last Ebola case is resolved on 14th September and 42 days later Nigeria is declared Ebola free.

At the end of the presentation there is the tone timeline for Nigeria in the studied period. The tone timeline graphically represents from extremely positive to extremely negative how emotions, in regards to one subject, change in time. It’s easily seen that when the first deaths of Ebola appear and Ebola is confirmed in Niger the articles that involve Nigeria tend to be more negative.

 

Full presentation can be seen here

Conclusions

Is there the certainty that all these mentions of suicides in Nigeria is linked to Ebola?

No. Other events could have influenced these mentions. This study was just a correlation created between the two events.