Saturday, May 31, 2014

Moving from traditional filesystem storage to ATMOS for Alfresco

I recently had the opportunity to configure Alfresco with ATMOS using the Alfresco S3 connector.

The company I work for has an ATMOS cloud setup across multiple data centers.

ATMOS is an object-based cloud storage platform to store, archive and access unstructured content at scale.

We had an existing instance of Alfresco leveraging NetApp storage that I had to migrate to ATMOS.

In order to do this I needed to download and install 2 tools.

ATMOS FOX - a firefox plugging that allows you to browse you ATMOS storage. https://addons.mozilla.org/en-US/firefox/addon/atmosfox/

AtmosSync.jar - a tool to copy files existing on a filesystem to ATMOS. https://code.google.com/p/atmos-java/downloads/detail?name=AtmosSync.jar&can=2&q=label%3AOpSys-All

The steps to make the move to ATMOS are fairly simple…

    1. Stop your Alfresco and Solr servers

    2. Create your S3 bucket with ATMOS FOX.
    3. After you create the bucket you must add a non listable meta tag (bucket-mapping-type=one_to_one) to the bucket folder.

     4. Copy your files in your contentstore and contentstore.deleted to Atmos using AtmosSync.jar.

     5. Update your alfresco-global.properties with your S3 configuration.
          ### S3 Config ###
          s3.accessKey=xxxxxxxxxxxx/xxxx
          s3.secretKey=xxxxxxxxs3.bucketName=bucketNAME
             #s3.bucketLocation=US
             s3.flatRoot=falses3service.https-only=false
             s3service.s3-endpoint=ATMOSHOST
             s3service.s3-endpoint-http-port=8080#
             s3service.disable-dns-buckets=false
             dir.contentstore=contentstore
             dir.contentstore.deleted=contentstore.deleted
          #Maximum disk usage for the cache in MB
             system.content.caching.maxUsageMB=51200
             #Maximum size of files which can be stored in the cache in MB (zero implies no limit)
             system.content.caching.maxFileSizeMB=0

    6. Back up your DB if you haven’t already

      7. Update all records in the ALF_CONTENT_URL table (store:// to s3://)
               UPDATE alf_content_url SET content_url = replace(content_url, 'store:', 's3:’)

   8. Startup Alfresco and Solr servers and you should be good to go.

Once you have verified that Alfresco is functioning properly you can repurpose the filesystem storage.

Wednesday, January 8, 2014

Duplicate entries put into HashSet Java issue

I have recently run into an issue using HashSet.addAll(Object) where duplicates are added to my Set.

I even made sure that the Object I was using Overrode hashCode().

Apparently the addAll() implementation for HashSet doesn't check for duplicates. This is bad because a reason for using a Set over a List is to avoid duplicate values.

If you loop through all of the values in the collection you are adding to the Set it works as expected not adding the duplicates. This is the approach I took to get around this issue.

for (item in myCollection){
set.add(item)
}

vs what I was trying to do which didn't work:

set.addAll(myCollection)

Another approach could be extend the HashSet and override the addAll() with the logic I used.

Saturday, June 22, 2013

JUnit TemporaryFolder Rule

If you need a temporary directory or file for testing and you are using jUnit, @Rule together with TemporaryFolder solves your problem.

The TemporaryFolder Rule allows creation of files and folders that are guaranteed to be deleted when the test method finishes (whether it passes or fails)

@Rule

public
TemporaryFolder tempFolder = new TemporaryFolder();

...

@Test
public void testFileCreation (){

File myFolder = tempFolder.newFile("test.);

fileGenerator.createFiles(2, myFolder);

myFolder.listFiles().length == 2;

}

The files that were generated will be deleted once the test finishes executing.

Wednesday, May 15, 2013

Using Groovy Closures To Simplify Working With Alfresco Transactions

If you are familiar with Alfresco transactions than you will be familiar with the following:

Boolean result = transactionService.getRetryingTransactionHelper().doInTransaction(new RetryingTransactionHelper.RetryingTransactionCallback<Boolean>() {
            public Boolean execute() throws Throwable {
                //Your code
               return true;
            }
}, false, true);
This is quite a bit of code and gets ugly if you start embedding a bunch of code inside this block. It is also not very testable with out some crazy mocking.

Groovy Closures to the rescue. http://groovy.codehaus.org/Closures
I created a simple TransactionHelper Groovy class with a method that takes a closure and wraps the closure inside of the above code required for a transaction and wired it as a Spring Bean.

import org.alfresco.repo.transaction.RetryingTransactionHelper
import org.alfresco.service.transaction.TransactionService
class TransactionHelper {
    private TransactionService transactionService
    def executeInSperarateTransaction(closure) {
        Boolean result = transactionService.getRetryingTransactionHelper().doInTransaction(new RetryingTransactionHelper.RetryingTransactionCallback<Boolean>() {
            public Boolean execute() throws Throwable {
                closure()
               return true
            }
        }, false, true)
    }
    void setTransactionService(TransactionService transactionService) {
        this.transactionService = transactionService
    }
}

Now if I need to execute a transaction I can simply do the following.
transactionHelper.executeInSperarateTransaction({ nodeService.addAspect(nodeRef, MyModel.THUMBNAIL_ASPECT, new HashMap<QName, Serializable>())})

You can simply pass a closure statement like above or pass in a closure like the following.

def closure() {
  def localVariable = new java.util.Date()
  return { println localVariable }
}
transactionHelper.executeInSeperateTransaction(closure)

For more information on Groovy and Closures see http://groovy.codehaus.org/Closures

Wednesday, April 10, 2013

Bootstraping Users with Alfresco

Bootstrapping data is a very common thing to do when deploying new modules. You may be expecting a certain taxonomy, certain set of rules, certain categories, certain users, certain metadata , etc. to be available to your new module. http://wiki.alfresco.com/wiki/Bootstrap_Data shows you how to bootstrap files, spaces, and categories. But I wanted to bootstrap a user that was going to be used to for a third-party system to interact with Alfresco. After some searching I came across http://forums.alfresco.com/forum/developer-discussions/repository-services/howto-bootstrap-users-09182008-1214 which described most of what I needed to do.

In order to bootstrap users you need to:

Create an xml file containing the person information (people.xml)

<?xml version="1.0" encoding="UTF-8"?>

<view:view xmlns:view="http://www.alfresco.org/view/repository/1.0" xmlns:alf="http://www.alfresco.org" xmlns:d="http://www.alfresco.org/model/dictionary/1.0" xmlns:sys="http://www.alfresco.org/model/system/1.0" xmlns:act="http://www.alfresco.org/model/action/1.0" xmlns:rule="http://www.alfresco.org/model/rule/1.0" xmlns:fm="http://www.alfresco.org/model/forum/1.0" xmlns:app="http://www.alfresco.org/model/application/1.0" xmlns:usr="http://www.alfresco.org/model/user/1.0" xmlns:ver="http://www.alfresco.org/model/versionstore/1.0" xmlns:cm="http://www.alfresco.org/model/content/1.0" xmlns="">

<cm:person view:childName="cm:0001">

<view:acl>

<view:ace view:access="ALLOWED">

<view:authority>jbarrett</view:authority>

<view:permission>All</view:permission>

</view:ace>

</view:acl>

<view:properties>

<cm:firstName>Josh</cm:firstName>

<cm:lastName>Barrett</cm:lastName>

<cm:email>jbarrett2k3@gmail.com</cm:email>

<cm:userName>jbarrett</cm:userName>

<cm:homeFolder>workspace://SpacesStore/MY_USERS_HOME</cm:homeFolder>

<cm:organizationId></cm:organizationId>

<cm:sizeQuota>-1</cm:sizeQuota>

<cm:sizeCurrent>0</cm:sizeCurrent>

</view:properties>

</cm:person>

</view:view>

Create an xml file containing the authority information (authorities.xml)

<?xml version="1.0" encoding="UTF-8"?>

<usr:user view:childName="usr:jbarrett">

<view:properties>

<usr:username>jbarrett</usr:username>

<usr:password>b02e3a7432ac716fdc2bb8df46ec5ab8</usr:password> <!—MD4 Hash which can be generated following the instructions http://wiki.alfresco.com/wiki/Security_and_Authentication#How_to_generate_the_correct_MD4_hash. -->

<usr:accountExpires>false</usr:accountExpires>

<usr:credentialsExpire>false</usr:credentialsExpire>

<usr:accountLocked>false</usr:accountLocked>

<usr:enabled>true</usr:enabled>

</view:properties>

</usr:user>

</view:view>

Bootstrap the xml files via a spring context file (mymodulebootstrap-context.xml)

<?xml version='1.0' encoding='UTF-8'?>

<!DOCTYPE beans PUBLIC '-//SPRING//DTD BEAN//EN' 'http://www.springframework.org/dtd/spring-beans.dtd'>

<beans>

<list>

<props>

<prop key="path">/${system.system_container.childname}/${system.people_container.childname}</prop>

<prop key="location">alfresco/module/myModule/bootstrap/people.xml</prop>

</props>

</list>

</property>

</bean>

<list>

<props>

<prop key="path">/${alfresco_user_store.system_container.childname}/${alfresco_user_store.user_container.childname}</prop>

<prop key="location">alfresco/module/mymodule/bootstrap/authorities.xml</prop>

</props>

</list>

</property>

</bean>

</beans>

That is all that is needed to bootstrap a user.

Tuesday, January 22, 2013

Cool Online Agile Board

http://leankit.com/

I stumbled across this tool today and it looks pretty neat.

The free account allows for 25 users and 10 boards.

Thursday, October 4, 2012

AMPs vs. JARs for Alfresco Customizations

From the alfresco wiki:

What is an AMP file?

An AMP file is a collection of code, XML, images, CSS, etc. that collectively extend the functionality or data provided by the standard Alfresco repository. For more information, see Developing an Alfresco Module.

An AMP file can contain as little as a set of custom templates or a new category. It can contain a custom model and associated UI customisations. It could contain a complete new set of functionality, for example records management.

As a general rule of thumb, anything that is considered to be an 'installable' extension to the Alfresco repository should be called a module and packaged as an AMP file.

AMP files can be installed into the Alfresco WAR using the Module Management Tool. An AMP file has a standard format described below which can be customised if required.

AMP files can now also be created via Apache Maven 2 archetypes and have their lifecycle and deployment into Alfresco managed by the Maven Alfresco AMP archetype. For a full description of Maven AMPs Alfresco support see Managing Alfresco Lifecyle with Maven.

Once the contents of the AMP file has been mapped into an Alfresco WAR using the Module Management Tool, the WAR can be deployed to the application server. When the repository is next started, the installed module configuration will be detected, and the repository will be bootstrapped to include the new module functionality and data.

A couple of problems I have with AMP files with custom code and configuration:

There is no easy way to back out an AMP extension
(Can be a little annoying if you are developing multiple AMPs that might be dependent on each other and files can be placed pretty much anywhere inside of the WAR File)
AMPs are not easy to work with Dependency Management.
(Major flaw in my eyes especially if you are trying to create base modules that may share configuration/code with other modules)

A couple of reasons I prefer to build my extensions with JAR files:

You don’t need any tools to deploy them
(The Jar files can simply be placed in the server lib directory or you can use tools like Maven to build the WAR file for you and automatically place the dependent JARs into the WEB-INF lib directory)
It is simple to back out an extension, simply remove the JAR file. You don’t need to roll back to the original WAR and reapply any other AMPs
The biggest reason I like to work with JAR files is I can work on multiple modules and leverage dependency management.

As far as I can tell you can do everything you can with JAR files in a much cleaner manner. You should still use AMP files but don't put custom code in them just pull in your dependent modules via Maven. The AMP will simply deploy your JARs into the WEB-INF/lib directory.

Here is a sample POM file you can use for an extension project:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

<groupId>com.sample.training</groupId>

<artifactId>alfrescoExtJar</artifactId>

<version>0.0.1-SNAPSHOT</version>

<name>Training Alfresco Extension</name>

<url>http://maven.apache.org</url>

<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>

</properties>

<id>releases</id>

<url>http://myNexus/nexus/content/repositories/releases/</url>

<uniqueVersion>false</uniqueVersion>

<layout>default</layout>

<name>My Nexus Repository</name>

</repository>

<id>snapshots</id>

<name>My Nexus Snapshot Repository</name>

<layout>default</layout>

<url>http://myNexus/nexus/content/repositories/snapshots/</url>

</snapshotRepository>

</distributionManagement>

<id>My Nexus Release</id>

<updatePolicy>always</updatePolicy>

</releases>

<enabled>false</enabled>

<updatePolicy>never</updatePolicy>

</snapshots>

<name>My Nexus</name>

<url>http://MyNexus/nexus/content/groups/public/</url>

</repository>

<id>My Nexus Snapshot</id>

<enabled>false</enabled>

<updatePolicy>never</updatePolicy>

</releases>

<updatePolicy>always</updatePolicy>

</snapshots>

<name>My Nexus Snapshots</name>

<url>http://MyNexus/nexus/content/groups/public-snapshots/</url>

</repository>

</repositories>

<groupId>junit</groupId>

<artifactId>junit</artifactId>

</dependency>

<groupId>org.mockito</groupId>

<artifactId>mockito-all</artifactId>

</dependency>

<groupId>org.alfresco.enterprise</groupId>

<artifactId>alfresco-core</artifactId>

<scope>provided</scope>

</dependency>

<groupId>org.alfresco.enterprise</groupId>

<artifactId>alfresco-repository</artifactId>

<scope>provided</scope>

</dependency>

... Other Provided Dependencies are not listed because it made the post too long...

</dependencies>

<build>

<finalName>alfrescoExtJar</finalName>

<artifactId>maven-compiler-plugin</artifactId>

<compilerId>groovy-eclipse-compiler</compilerId>

</configuration>

<groupId>org.codehaus.groovy</groupId>

<artifactId>groovy-eclipse-compiler</artifactId>

<groupId>org.codehaus.groovy</groupId>

<artifactId>groovy-eclipse-batch</artifactId>

</exclusion>

</exclusions>

</dependency>

<groupId>org.codehaus.groovy</groupId>

<artifactId>groovy-eclipse-batch</artifactId>

</dependency>

</dependencies>

</plugin>

</plugins>

</build>

</project>

This POM file is quite length because I have included all of the Alfresco Provided dependencies. You could clean this up by putting them in a separate POM file and adding that as a dependency.

<groupId>com.sample.commons.alfresco</groupId>

<artifactId>alfresco-include</artifactId>

<scope>provided</scope>

</dependency>

If you had other reusable alfresco libraries you can include them by just inserting another dependency…

<dependency>

<groupId>com.sample.training</groupId>

<artifactId>shared</artifactId>

<version>0.0.1-SNAPSHOT</version>

<scope>compile</scope>

</dependency>

This is great and all but how does this work in my IDE if I want to run this. You can add a maven webapp project to your IDE and use a POM similar to this one:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">

<groupId>com.sample.training</groupId>

<artifactId>alfresco</artifactId>

<version>0.0.1-SNAPSHOT</version>

<name>Training Alfresco Webapp</name>

<url>http://maven.apache.org</url>

<id>releases</id>

<url>http://myNexus/nexus/content/repositories/releases/</url>

<uniqueVersion>false</uniqueVersion>

<layout>default</layout>

<name>My Nexus Repository</name>

</repository>

<id>snapshots</id>

<name>My Nexus Snapshot Repository</name>

<layout>default</layout>

<url>http://myNexus/nexus/content/repositories/snapshots/</url>

</snapshotRepository>

</distributionManagement>

<id>My Nexus Release</id>

<updatePolicy>always</updatePolicy>

</releases>

<enabled>false</enabled>

<updatePolicy>never</updatePolicy>

</snapshots>

<name>My Nexus</name>

<url>http://MyNexus/nexus/content/groups/public/</url>

</repository>

<id>My Nexus Snapshot</id>

<enabled>false</enabled>

<updatePolicy>never</updatePolicy>

</releases>

<updatePolicy>always</updatePolicy>

</snapshots>

<name>My Nexus Snapshots</name>

<url>http://MyNexus/nexus/content/groups/public-snapshots/</url>

</repository>

</repositories>

<groupId>org.alfresco.enterprise</groupId>

<artifactId>alfresco</artifactId>

</dependency>

<groupId>com.sample.training</groupId>

<artifactId>alfrescoExtJar</artifactId>

<version>0.0.1-SNAPSHOT</version>

<scope>runtime</scope>

</dependency>

</dependencies>

<build>

<finalName>alfresco</finalName>

</build>

</project>

Include the alfresco WAR as a dependency and add you Ext dependencies. This will build you a WAR file with your JARs place in the lib directory. If you are using eclipse you can leverage WTP and deploy to a tomcat server. You can do the same technique with share.

This technique has helped to get teams up and running in minutes. If you hook this up with a CI system like Jenkins you will have a well-oiled machine building your artifacts that can easily be deployed. You can now deploy the Jar files and also deploy prebuilt WAR files. When you need to test you customizations out against another version of Alfresco, simply change a version # in your POM files and try it out.

When you are ready to deploy build an AMP with no custom code or configuration. Just pull in the dependencies. This will build and AMP file that basically only contains Jar files.