Why am I seeing error "All remote hosts failed" when trying to retrieve content?

Below are things to check when seeing error "Failed to transfer content due to a transport error com.documentum.ucf.common.UCFException: All remote hosts failed":

Background:
When using ACS to retrieve content (the default content transfer mode starting in 5.3 SP1), DFC produces URLs that point to content on the ACS/Content Server machine. UCF on the browser machine tries to use these URLs to retrieve the content. These URLs are produced based on a "base url" in the ACS config attribute. (The ACS config is managed in DA.) This means the browser machines requires connecting to the ACS/Content Server machine.

A URL will not be generated for an ACS that is not running but if ACS goes down AFTER the URL was generated that client will fail with the error but all other content requests will succeed using the pre-ACS content transfer mode (just using Content Server) until ACS is restarted.

1. One possibility for the error is that URLs cannot be DNS-resolved on a client computer. (like http://localhost, or http://servername instead of http://servername.emc.com). If need be, the base URL can be changed using IAPI:

API>retrieve,c,dm_acs_config
API>set,c,l,acs_base_url
SET>
API>save,c,l

In order to check which URL the browser is using to access content, change the UCF log level on a client side (where the browser is) by changing:

.level=FINE

In the file:

C:\Documents and Settings\\Documentum\ucf\\shared\config\ucf.client.logging.properties

The output goes to file- C:\Documentum\Logs\ucf.trace<….>.log

Another way is to get the same information on a Webtop/DFC side. Enable INFO or DEBUG logging for a category

com.documentum.acs

Webtop machine in the log4j.properties file. These changes should be picked up dynamically.

2. Check that a firewall does not restrict access to these URLs from the client computer.

3. Check the $DOCUMENTUM/shared/logs/AcsServer.log for errors.

4. If the following error occurs in the Java Method Server log:

FATAL [HttpProcessor[9080][4]] acs.com.documentum.acs.sdi.ACS -
java.lang.NoSuchMethodError: org.aspectj.runtime.reflect.Factory.makeJP(Lorg/aspectj/lang/JoinPoint$StaticPart;Ljava/lan
g/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)Lorg/aspectj/lang/JoinPoint;

and DCM is being used in the Docbase, go to C:\Program Files\Documentum\tcf\lib or the appropriate place on the UNIX system and rename file aspectjrt1.1.0.jar to something like aspectjrt1.1.0.bak

In some cases, you may be getting the "all remote hosts failed" error in a standalone docbase WITHOUT DCM, and in that case, you will need to go find the 1.2.0 version of that jar file, aspectjrt-1.2.0.jar, merged with custom documentum classes in the Documentum\product\5.3\tomcat\webapps\ACS\WEB-INF\lib\ucf-server-impl.jar class.
If you copy that the aspectjrt-1.2.0.jar to the lib directory and rename the aspectjrt-1.1.0.jar to something like .bak, then all errors go away.

5. If there is an error like 'java.lang.NoClassDefFoundError: org/aspectj/lang/NoAspectBoundException', the reason for this error is that Application Builder is installed on the same machine as the Content Server.

App Builder and ACS server use common jar file ucf-server-api.jar. In the classpath for ACS Server, the ucf-server-api.jar located in program Files\Application Builder\lib directory has precedence over the same jar file present in ACS server webapp lib directory. Quick test to confirm this would be rename the ucf-server-api.jar in Application Builder lib directory to ucf-server-api.bak and restart the Java Method Server. Note that you have to stop the method server before renaming the jar file as the jar will be in use while the server is running and will not allow you to rename the file.


The workaround for this is to install Application Builder on a different machine or modify the classpath of the java method Server so that App Builders ucf-server-api.jar is not in the classpath of the java method server.