In this article we will learn File Connector in MuleSoft message flow along with working examples.
file:read – It obtains the content and meta data of a file at a given path.
file:list – It lists all the files in a given directory.
file:matcher with directories=”EXCLUDE” and regularFiles=”REQUIRE” under file:list we may use for excluding folder in message flow.
file:matcher with directories=”REQUIRE” under file:list we may use for excluding files in message flow.
We may use recursive=”false” under file:list for skipping sub folder in message flow.
file:move – We may use this file operation for moving a file from source directory to target directory in message flow.
When the file is read from the FTP source folder, the default behavior of the operation in Mule 4 is that it remains in the same folder until it is manually deleted.
Here, is the MuleSoft project structure. In this working example we are exposing file operations as REST APIs.
REST API Operation available:
/readfile – This Rest API obtains the content and meta data of a file at a given path.
/listall – This Rest API lists all the files in a given directory.
/listfiles – This Rest API lists files only in a given directory and sub folders.
/listdir – This Rest API lists directories only in a given directory and sub folders without files.
/listnonrecfiles – This Rest API lists files in a given directory not in its sub folders.
/move – This Rest API moves a file from source directory to destination directory.
/delete – This Rest API deletes a file from the directory.
/createdir – This Rest API creates new directory.
/copy – This Rest API copy a file/directory from source to destination.
Following Mule Flows file contains multiple mule message flows for different file operations.
<?xml version="1.0" encoding="UTF-8"?> <mule xmlns:ee="http://www.mulesoft.org/schema/mule/ee/core" xmlns:http="http://www.mulesoft.org/schema/mule/http" xmlns:file="http://www.mulesoft.org/schema/mule/file" xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" http://www.mulesoft.org/schema/mule/ee/core http://www.mulesoft.org/schema/mule/ee/core/current/mule-ee.xsd http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd http://www.mulesoft.org/schema/mule/file http://www.mulesoft.org/schema/mule/file/current/mule-file.xsd http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd"> <http:listener-config name="HTTP_Listener_config" doc:name="HTTP Listener config" doc:id="d8a12d90-53fd-46cf-bec1-d332add14b80"> <http:listener-connection host="0.0.0.0" port="8085" /> </http:listener-config> <file:config name="File_Config_Resources" doc:name="File Config" doc:id="02ba071c-e57d-400c-aa69-4148a873e620"> <file:connection workingDir="D:\development\AnypointStudio\studio-workspace\mule-file-connector-example\src\test\resources\data" /> </file:config> <file:config name="File_Config_Resources_Output" doc:name="File Config" doc:id="dd86d135-8a44-4b0f-8dfd-28a1584331e8"> <file:connection workingDir="D:\development\AnypointStudio\studio-workspace\mule-file-connector-example\src\test\resources\data\output" /> </file:config> <flow name="read_file_flow" doc:id="3154190b-997d-4a97-a551-f9d0800bb380"> <http:listener doc:name="Listener" doc:id="79c7ef22-136b-4203-8078-728fe0f1cbed" config-ref="HTTP_Listener_config" path="/readfile" /> <file:read doc:name="read" doc:id="0df1c1b6-32d0-43b9-b3b8-5c2509ed955m" path="D:\development\AnypointStudio\studio-workspace\mule-file-connector-example\src\test\resources\data\input\new\file.txt" config-ref="File_Config_Resources" lock="true" /> </flow> <!-- List all the files and folder information from a given directory --> <!-- List all the files information (without folder) from a given directory --> <flow name="list_files_and_directories_flow" doc:id="282f2636-661e-45b5-9224-da3443935cb6"> <http:listener doc:name="Listener" doc:id="226cf0f5-e39f-4e91-bcb0-b614a163716e" config-ref="HTTP_Listener_config" path="/listall" /> <file:list doc:name="List" doc:id="96832185-d0ba-456c-8a58-22704ac644dd" directoryPath="D:\development\AnypointStudio\studio-workspace\mule-file-connector-example\src\test\resources\data" config-ref="File_Config_Resources" recursive="true"> <file:matcher /> </file:list> <ee:transform doc:name="list-files-and-directories-payload" doc:id="0315ae33-9949-4cf0-86fc-f8c6d6b2d40f"> <ee:message> <ee:set-payload><![CDATA[%dw 2.0 output application/json --- payload map((item,index) -> { fileName:item.attributes.fileName, fileCreateTimeStamp: item.attributes.creationTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"}, filePath:item.attributes.path, fileLastModified: item.attributes.lastModifiedTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"}, }) ]]></ee:set-payload> </ee:message> </ee:transform> <logger level="INFO" doc:name="Logger" doc:id="aeed72db-38c2-40b1-87dd-7a208d60760a" message="List all the files and folder information from a given directory done!!" /> </flow> <flow name="list_files_only_flow" doc:id="ad3fd987-3481-474e-ae6c-753e2f41a787"> <http:listener doc:name="Listener" doc:id="1425e119-4950-4150-9c68-fd0cdec42368" config-ref="HTTP_Listener_config" path="/listfiles" /> <file:list doc:name="List" doc:id="bcb27ff3-0771-41f8-ac9d-e15a953bb0dd" directoryPath="D:\development\AnypointStudio\studio-workspace\mule-file-connector-example\src\test\resources\data" config-ref="File_Config_Resources" recursive="true"> <file:matcher directories="EXCLUDE" regularFiles="REQUIRE" /> </file:list> <ee:transform doc:name="list-files-only-payload" doc:id="dbb38e5c-01ad-4a5a-ab06-5b0a823017ea"> <ee:message> <ee:set-payload><![CDATA[%dw 2.0 output application/json --- payload map((item,index) -> { fileName:item.attributes.fileName, fileCreateTimeStamp: item.attributes.creationTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"}, filePath:item.attributes.path, fileLastModified: item.attributes.lastModifiedTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"}, }) ]]></ee:set-payload> </ee:message> </ee:transform> <logger level="INFO" doc:name="Logger" doc:id="27aba641-93e4-486b-97a8-025663375b33" message="List all the files information (without folder) from a given directory done!!" /> </flow> <!-- List all the directories/folder information(without files) --> <flow name="list_directory_only_flow" doc:id="ce388f0b-c813-4c27-9a04-27c750ac3458"> <http:listener doc:name="Listener" doc:id="9f6df059-6c03-403c-a706-1fc5f0936262" config-ref="HTTP_Listener_config" path="/listdir" /> <file:list doc:name="List" doc:id="18627526-71a2-4bc5-b251-b3e0bbbc2d6d" directoryPath="D:\development\AnypointStudio\studio-workspace\mule-file-connector-example\src\test\resources\data" config-ref="File_Config_Resources" recursive="true"> <file:matcher directories="REQUIRE" /> </file:list> <ee:transform doc:name="list-directory-only-payload" doc:id="96287b83-b078-46de-9e60-49ec68709192"> <ee:message> <ee:set-payload><![CDATA[%dw 2.0 output application/json --- payload map((item,index) -> { fileName:item.attributes.fileName, fileCreateTimeStamp: item.attributes.creationTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"}, filePath:item.attributes.path, fileLastModified: item.attributes.lastModifiedTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"}, }) ]]></ee:set-payload> </ee:message> </ee:transform> <logger level="INFO" doc:name="Logger" doc:id="9aeeaea5-a1ec-454c-ad48-5b8c97088399" message="List all the directories/folder information(without files) done!!" /> </flow> <!-- List all the file or directories information irrespective of any sub folder from a given path --> <flow name="list_files_non_recursive_flow" doc:id="812b4558-9114-4829-b140-40cf06035490"> <http:listener doc:name="Listener" doc:id="67b676bc-5e7f-4f1b-8bda-88228998c974" config-ref="HTTP_Listener_config" path="/listnonrecfiles" /> <file:list doc:name="List" doc:id="95f77f70-b472-404c-b36a-cba265c37267" directoryPath="D:\development\AnypointStudio\studio-workspace\mule-file-connector-example\src\test\resources\data\input" config-ref="File_Config_Resources" recursive="false"> <file:matcher /> </file:list> <ee:transform doc:name="list-files-non-recursive-payload" doc:id="01b80b13-5c12-4e17-a033-9ef07651a9ae"> <ee:message> <ee:set-payload><![CDATA[%dw 2.0 output application/json --- payload map((item,index) -> { fileName:item.attributes.fileName, fileCreateTimeStamp: item.attributes.creationTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"}, filePath:item.attributes.path, fileLastModified: item.attributes.lastModifiedTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"}, }) ]]></ee:set-payload> </ee:message> </ee:transform> <logger level="INFO" doc:name="Logger" doc:id="a21afae5-3c9e-4204-9446-a04d9df7c313" message="List all the file or directories information irrespective of any subfolder from a given path done!!" /> </flow> <flow name="move_file_flow" doc:id="750ca921-e79f-4b60-b448-9b2801ae9df7"> <http:listener doc:name="Listener" doc:id="36277dc4-5454-4ed8-bd9c-0061d05b8ba0" config-ref="HTTP_Listener_config" path="/move" /> <file:move doc:name="Move" doc:id="f8df27a5-a3d9-45ba-baca-b8e507c6f61c" sourcePath="D:\development\AnypointStudio\studio-workspace\mule-file-connector-example\src\test\resources\data\input\processing\file2.txt" targetPath="D:\development\AnypointStudio\studio-workspace\mule-file-connector-example\src\test\resources\output" config-ref="File_Config_Resources"> <reconnect /> </file:move> </flow> <flow name="file_delete_flow" doc:id="e3712fc3-f3af-43ff-994f-145b3cda026f"> <http:listener doc:name="Listener" doc:id="e2dec67d-8efe-4152-8c23-016ad250a892" config-ref="HTTP_Listener_config" path="/delete" /> <file:delete doc:name="Delete" doc:id="903734d0-40e6-46f0-91f5-580250622488" config-ref="File_Config_Resources" path="D:\development\AnypointStudio\studio-workspace\mule-file-connector-example\src\test\resources\data\output" /> </flow> <flow name="file_create_directory_flow" doc:id="dc7080f4-1ca6-4394-a3e9-f2f854d5937b"> <http:listener doc:name="Listener" doc:id="f44cbd58-3e93-4453-9436-60f2de3cc22d" config-ref="HTTP_Listener_config" path="/createdir" /> <file:create-directory doc:name="Create directory" doc:id="6ef6dab7-dce9-44b8-8dcd-c0eb915338dd" config-ref="File_Config_Resources" directoryPath="D:\development\AnypointStudio\studio-workspace\mule-file-connector-example\src\test\resources\data\failed" /> </flow> <flow name="file_copy_flow" doc:id="031391f5-d786-4212-8154-84fd59a41832"> <http:listener doc:name="Listener" doc:id="55177491-15e9-423d-946c-1b02797562d8" config-ref="HTTP_Listener_config" path="/copy" /> <file:copy doc:name="Copy" doc:id="6a825bc1-bfb5-41a5-ac82-d67ee69b083f" sourcePath="D:\development\AnypointStudio\studio-workspace\mule-file-connector-example\src\test\resources\data\input\processing\" targetPath="D:\development\AnypointStudio\studio-workspace\mule-file-connector-example\src\test\resources\data\failed" config-ref="File_Config_Resources" /> </flow> </mule>
Locking a file is a mechanism in which a file get locked for other operation so that no one can access file when the processing is going on.
E.g. In the following message flow we are locking the file.txt during read operation.
<flow name="read_file_flow" doc:id="3154190b-997d-4a97-a551-f9d0800bb380"> <http:listener doc:name="Listener" doc:id="79c7ef22-136b-4203-8078-728fe0f1cbed" config-ref="HTTP_Listener_config" path="/readfile" /> <file:read doc:name="read" doc:id="0df1c1b6-32d0-43b9-b3b8-5c2509ed955m" path="D:\development\AnypointStudio\studio-workspace\mule-file-connector-example\src\test\resources\data\input\new\file.txt" config-ref="File_Config_Resources" lock="true" /> </flow>
Drag and drop file read operation from mule palette and configure the file path to read the file.
<http:listener-config name="HTTP_Listener_config" doc:name="HTTP Listener config" doc:id="d8a12d90-53fd-46cf-bec1-d332add14b80"> <http:listener-connection host="0.0.0.0" port="8085" /> </http:listener-config> <file:config name="File_Config_Resources" doc:name="File Config" doc:id="02ba071c-e57d-400c-aa69-4148a873e620"> <file:connection workingDir="D:\development\AnypointStudio\studio-workspace\mule-file-connector-example\src\test\resources\data" /> </file:config> <file:config name="File_Config_Resources_Output" doc:name="File Config" doc:id="dd86d135-8a44-4b0f-8dfd-28a1584331e8"> <file:connection workingDir="D:\development\AnypointStudio\studio-workspace\mule-file-connector-example\src\test\resources\data\output" /> </file:config> <flow name="read_file_flow" doc:id="3154190b-997d-4a97-a551-f9d0800bb380"> <http:listener doc:name="Listener" doc:id="79c7ef22-136b-4203-8078-728fe0f1cbed" config-ref="HTTP_Listener_config" path="/readfile" /> <file:read doc:name="read" doc:id="0df1c1b6-32d0-43b9-b3b8-5c2509ed955m" path="D:\development\AnypointStudio\studio-workspace\mule-file-connector-example\src\test\resources\data\input\new\file.txt" config-ref="File_Config_Resources" lock="true" /> </flow>
********************************************************************** INFO 2021-08-10 18:49:58,995 [WrapperListener_start_runner] org.mule.runtime.core.internal.logging.LogUtil: ********************************************************************** * Started app 'mule-file-connector-example' * * Application plugins: * * - File Common Plugin : 1.3.2 * * - Sockets : 1.1.5 * * - File : 1.3.2 * * - HTTP : 1.5.11 * ********************************************************************** INFO 2021-08-10 18:49:59,058 [WrapperListener_start_runner] org.mule.runtime.core.internal.logging.LogUtil: ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + Mule is up and kicking (every 5000ms) + ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ INFO 2021-08-10 18:49:59,087 [WrapperListener_start_runner] org.eclipse.jetty.server.AbstractConnector: Started ServerConnector@1619078f{HTTP/1.1,[http/1.1]}{0.0.0.0:59321} INFO 2021-08-10 18:49:59,091 [WrapperListener_start_runner] org.mule.runtime.core.internal.logging.LogUtil: ********************************************************************** * - - + DOMAIN + - - * - - + STATUS + - - * ********************************************************************** * default * DEPLOYED * ********************************************************************** ******************************************************************************************************* * - - + APPLICATION + - - * - - + DOMAIN + - - * - - + STATUS + - - * ******************************************************************************************************* * mule-file-connector-example * default * DEPLOYED * ******************************************************************************************************* INFO 2021-08-10 22:11:11,528 [http.listener.01 SelectorRunner] org.mule.service.http.impl.service.util.DefaultRequestMatcherRegistry: No listener found for request: (GET)/read INFO 2021-08-10 22:11:11,564 [http.listener.01 SelectorRunner] org.mule.service.http.impl.service.util.DefaultRequestMatcherRegistry: Available listeners are: [([*])/delete/, ([*])/readfile/, ([*])/createdir/, ([*])/listfiles/, ([*])/move/, ([*])/copy/, ([*])/listdir/, ([*])/listall/, ([*])/listnonrecfiles/]
If you want to retrieve both files and folder information present at the given path, use List connector with file matcher rules as “Directories ” and “Regular files” values as “INCLUDES” which is default configuration as below.
<!-- List all the files and folder information from a given directory --> <flow name="list_files_and_directories_flow" doc:id="282f2636-661e-45b5-9224-da3443935cb6"> <http:listener doc:name="Listener" doc:id="226cf0f5-e39f-4e91-bcb0-b614a163716e" config-ref="HTTP_Listener_config" path="/listall" /> <file:list doc:name="List" doc:id="96832185-d0ba-456c-8a58-22704ac644dd" directoryPath="D:\development\AnypointStudio\studio-workspace\mule-file-connector-example\src\test\resources\data" config-ref="File_Config_Resources" recursive="true"> <file:matcher /> </file:list> <ee:transform doc:name="list-files-and-directories-payload" doc:id="0315ae33-9949-4cf0-86fc-f8c6d6b2d40f"> <ee:message> <ee:set-payload><![CDATA[%dw 2.0 output application/json --- payload map((item,index) -> { fileName:item.attributes.fileName, fileCreateTimeStamp: item.attributes.creationTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"}, filePath:item.attributes.path, fileLastModified: item.attributes.lastModifiedTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"}, }) ]]></ee:set-payload> </ee:message> </ee:transform> <logger level="INFO" doc:name="Logger" doc:id="aeed72db-38c2-40b1-87dd-7a208d60760a" message="List all the files and folder information from a given directory done!!" /> </flow>
INFO 2021-08-10 18:49:58,995 [WrapperListener_start_runner] org.mule.runtime.core.internal.logging.LogUtil: ********************************************************************** * Started app 'mule-file-connector-example' * * Application plugins: * * - File Common Plugin : 1.3.2 * * - Sockets : 1.1.5 * * - File : 1.3.2 * * - HTTP : 1.5.11 * ********************************************************************** INFO 2021-08-10 18:49:59,058 [WrapperListener_start_runner] org.mule.runtime.core.internal.logging.LogUtil: ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + Mule is up and kicking (every 5000ms) + ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ INFO 2021-08-10 18:49:59,087 [WrapperListener_start_runner] org.eclipse.jetty.server.AbstractConnector: Started ServerConnector@1619078f{HTTP/1.1,[http/1.1]}{0.0.0.0:59321} INFO 2021-08-10 18:49:59,091 [WrapperListener_start_runner] org.mule.runtime.core.internal.logging.LogUtil: ********************************************************************** * - - + DOMAIN + - - * - - + STATUS + - - * ********************************************************************** * default * DEPLOYED * ********************************************************************** ******************************************************************************************************* * - - + APPLICATION + - - * - - + DOMAIN + - - * - - + STATUS + - - * ******************************************************************************************************* * mule-file-connector-example * default * DEPLOYED * ******************************************************************************************************* INFO 2021-08-10 22:11:11,528 [http.listener.01 SelectorRunner] org.mule.service.http.impl.service.util.DefaultRequestMatcherRegistry: No listener found for request: (GET)/read INFO 2021-08-10 22:11:11,564 [http.listener.01 SelectorRunner] org.mule.service.http.impl.service.util.DefaultRequestMatcherRegistry: Available listeners are: [([*])/delete/, ([*])/readfile/, ([*])/createdir/, ([*])/listfiles/, ([*])/move/, ([*])/copy/, ([*])/listdir/, ([*])/listall/, ([*])/listnonrecfiles/] INFO 2021-08-10 22:20:51,134 [[MuleRuntime].cpuIntensive.01: [mule-file-connector-example].list_files_and_directories_flow.CPU_INTENSIVE @676a3901] [event: 1e7629a0-f9fb-11eb-a30a-a44cc84ce10c] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: List all the files and folder information from a given directory done!!
If you specifically want to retrieve only list of files(excluding folder information ) present at any given path, use List connector with file matcher rules as “Directories” values as “EXCLUDES”and “Regular files” values as “REQUIRE” during connector configuration.
<!-- List all the files information (without folder) from a given directory --> <flow name="list_files_only_flow" doc:id="ad3fd987-3481-474e-ae6c-753e2f41a787"> <http:listener doc:name="Listener" doc:id="1425e119-4950-4150-9c68-fd0cdec42368" config-ref="HTTP_Listener_config" path="/listfiles" /> <file:list doc:name="List" doc:id="bcb27ff3-0771-41f8-ac9d-e15a953bb0dd" directoryPath="D:\development\AnypointStudio\studio-workspace\mule-file-connector-example\src\test\resources\data" config-ref="File_Config_Resources" recursive="true"> <file:matcher directories="EXCLUDE" regularFiles="REQUIRE" /> </file:list> <ee:transform doc:name="list-files-only-payload" doc:id="dbb38e5c-01ad-4a5a-ab06-5b0a823017ea"> <ee:message> <ee:set-payload><![CDATA[%dw 2.0 output application/json --- payload map((item,index) -> { fileName:item.attributes.fileName, fileCreateTimeStamp: item.attributes.creationTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"}, filePath:item.attributes.path, fileLastModified: item.attributes.lastModifiedTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"}, }) ]]></ee:set-payload> </ee:message> </ee:transform> <logger level="INFO" doc:name="Logger" doc:id="27aba641-93e4-486b-97a8-025663375b33" message="List all the files information (without folder) from a given directory done!!" /> </flow>
********************************************************************** INFO 2021-08-10 18:49:58,995 [WrapperListener_start_runner] org.mule.runtime.core.internal.logging.LogUtil: ********************************************************************** * Started app 'mule-file-connector-example' * * Application plugins: * * - File Common Plugin : 1.3.2 * * - Sockets : 1.1.5 * * - File : 1.3.2 * * - HTTP : 1.5.11 * ********************************************************************** INFO 2021-08-10 18:49:59,058 [WrapperListener_start_runner] org.mule.runtime.core.internal.logging.LogUtil: ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + Mule is up and kicking (every 5000ms) + ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ INFO 2021-08-10 18:49:59,087 [WrapperListener_start_runner] org.eclipse.jetty.server.AbstractConnector: Started ServerConnector@1619078f{HTTP/1.1,[http/1.1]}{0.0.0.0:59321} INFO 2021-08-10 18:49:59,091 [WrapperListener_start_runner] org.mule.runtime.core.internal.logging.LogUtil: ********************************************************************** * - - + DOMAIN + - - * - - + STATUS + - - * ********************************************************************** * default * DEPLOYED * ********************************************************************** ******************************************************************************************************* * - - + APPLICATION + - - * - - + DOMAIN + - - * - - + STATUS + - - * ******************************************************************************************************* * mule-file-connector-example * default * DEPLOYED * ******************************************************************************************************* INFO 2021-08-10 22:11:11,528 [http.listener.01 SelectorRunner] org.mule.service.http.impl.service.util.DefaultRequestMatcherRegistry: No listener found for request: (GET)/read INFO 2021-08-10 22:11:11,564 [http.listener.01 SelectorRunner] org.mule.service.http.impl.service.util.DefaultRequestMatcherRegistry: Available listeners are: [([*])/delete/, ([*])/readfile/, ([*])/createdir/, ([*])/listfiles/, ([*])/move/, ([*])/copy/, ([*])/listdir/, ([*])/listall/, ([*])/listnonrecfiles/] INFO 2021-08-10 22:20:51,134 [[MuleRuntime].cpuIntensive.01: [mule-file-connector-example].list_files_and_directories_flow.CPU_INTENSIVE @676a3901] [event: 1e7629a0-f9fb-11eb-a30a-a44cc84ce10c] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: List all the files and folder information from a given directory done!! INFO 2021-08-10 22:26:48,168 [[MuleRuntime].cpuIntensive.03: [mule-file-connector-example].list_files_only_flow.CPU_INTENSIVE @6c98400b] [event: f36da9d1-f9fb-11eb-a30a-a44cc84ce10c] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: List all the files information (without folder) from a given directory done!!
If you specifically want to retrieve only list of folder(excluding files information) present at the given path, use List connector with file matcher rules as “Directories” values as “REQUIRE” and “Regular files” values as “EXCLUDES” during connector configuration.
<!-- List all the directories/folder information(without files) --> <flow name="list_directory_only_flow" doc:id="ce388f0b-c813-4c27-9a04-27c750ac3458"> <http:listener doc:name="Listener" doc:id="9f6df059-6c03-403c-a706-1fc5f0936262" config-ref="HTTP_Listener_config" path="/listdir" /> <file:list doc:name="List" doc:id="18627526-71a2-4bc5-b251-b3e0bbbc2d6d" directoryPath="D:\development\AnypointStudio\studio-workspace\mule-file-connector-example\src\test\resources\data" config-ref="File_Config_Resources" recursive="true"> <file:matcher directories="REQUIRE" /> </file:list> <ee:transform doc:name="list-directory-only-payload" doc:id="96287b83-b078-46de-9e60-49ec68709192"> <ee:message> <ee:set-payload><![CDATA[%dw 2.0 output application/json --- payload map((item,index) -> { fileName:item.attributes.fileName, fileCreateTimeStamp: item.attributes.creationTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"}, filePath:item.attributes.path, fileLastModified: item.attributes.lastModifiedTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"}, }) ]]></ee:set-payload> </ee:message> </ee:transform> <logger level="INFO" doc:name="Logger" doc:id="9aeeaea5-a1ec-454c-ad48-5b8c97088399" message="List all the directories/folder information(without files) done!!" /> </flow>
********************************************************************** INFO 2021-08-10 18:49:58,995 [WrapperListener_start_runner] org.mule.runtime.core.internal.logging.LogUtil: ********************************************************************** * Started app 'mule-file-connector-example' * * Application plugins: * * - File Common Plugin : 1.3.2 * * - Sockets : 1.1.5 * * - File : 1.3.2 * * - HTTP : 1.5.11 * ********************************************************************** INFO 2021-08-10 18:49:59,058 [WrapperListener_start_runner] org.mule.runtime.core.internal.logging.LogUtil: ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + Mule is up and kicking (every 5000ms) + ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ INFO 2021-08-10 18:49:59,087 [WrapperListener_start_runner] org.eclipse.jetty.server.AbstractConnector: Started ServerConnector@1619078f{HTTP/1.1,[http/1.1]}{0.0.0.0:59321} INFO 2021-08-10 18:49:59,091 [WrapperListener_start_runner] org.mule.runtime.core.internal.logging.LogUtil: ********************************************************************** * - - + DOMAIN + - - * - - + STATUS + - - * ********************************************************************** * default * DEPLOYED * ********************************************************************** ******************************************************************************************************* * - - + APPLICATION + - - * - - + DOMAIN + - - * - - + STATUS + - - * ******************************************************************************************************* * mule-file-connector-example * default * DEPLOYED * ******************************************************************************************************* INFO 2021-08-10 22:11:11,528 [http.listener.01 SelectorRunner] org.mule.service.http.impl.service.util.DefaultRequestMatcherRegistry: No listener found for request: (GET)/read INFO 2021-08-10 22:11:11,564 [http.listener.01 SelectorRunner] org.mule.service.http.impl.service.util.DefaultRequestMatcherRegistry: Available listeners are: [([*])/delete/, ([*])/readfile/, ([*])/createdir/, ([*])/listfiles/, ([*])/move/, ([*])/copy/, ([*])/listdir/, ([*])/listall/, ([*])/listnonrecfiles/] INFO 2021-08-10 22:20:51,134 [[MuleRuntime].cpuIntensive.01: [mule-file-connector-example].list_files_and_directories_flow.CPU_INTENSIVE @676a3901] [event: 1e7629a0-f9fb-11eb-a30a-a44cc84ce10c] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: List all the files and folder information from a given directory done!! INFO 2021-08-10 22:26:48,168 [[MuleRuntime].cpuIntensive.03: [mule-file-connector-example].list_files_only_flow.CPU_INTENSIVE @6c98400b] [event: f36da9d1-f9fb-11eb-a30a-a44cc84ce10c] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: List all the files information (without folder) from a given directory done!! INFO 2021-08-10 22:36:01,158 [[MuleRuntime].cpuIntensive.02: [mule-file-connector-example].list_directory_only_flow.CPU_INTENSIVE @2a63315a] [event: 3d0f0a11-f9fd-11eb-a30a-a44cc84ce10c] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: List all the directories/folder information(without files) done!!
If you specifically want to retrieve all the files /directories/ only files/ only directories information irrespective of any subfolders present at any given location, use List connector with the below connector configuration.
If you put “Recursive” value as “True” — It will give you all the list of files present in current folder and all subfolders available on the given path.
Incase you put “Recursive” value as “False ” — This is a default configuration . It will collect all the list of files present in the current directory . It doesn’t look for files present in any of the subfolders available on the given path.
<!-- List all the file or directories information irrespective of any sub folder from a given path --> <flow name="list_files_non_recursive_flow" doc:id="812b4558-9114-4829-b140-40cf06035490"> <http:listener doc:name="Listener" doc:id="67b676bc-5e7f-4f1b-8bda-88228998c974" config-ref="HTTP_Listener_config" path="/listnonrecfiles" /> <file:list doc:name="List" doc:id="95f77f70-b472-404c-b36a-cba265c37267" directoryPath="D:\development\AnypointStudio\studio-workspace\mule-file-connector-example\src\test\resources\data\input" config-ref="File_Config_Resources" recursive="false"> <file:matcher /> </file:list> <ee:transform doc:name="list-files-non-recursive-payload" doc:id="01b80b13-5c12-4e17-a033-9ef07651a9ae"> <ee:message> <ee:set-payload><![CDATA[%dw 2.0 output application/json --- payload map((item,index) -> { fileName:item.attributes.fileName, fileCreateTimeStamp: item.attributes.creationTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"}, filePath:item.attributes.path, fileLastModified: item.attributes.lastModifiedTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"}, }) ]]></ee:set-payload> </ee:message> </ee:transform> <logger level="INFO" doc:name="Logger" doc:id="a21afae5-3c9e-4204-9446-a04d9df7c313" message="List all the file or directories information irrespective of any subfolder from a given path done!!" /> </flow>
INFO 2021-08-10 18:49:58,995 [WrapperListener_start_runner] org.mule.runtime.core.internal.logging.LogUtil: ********************************************************************** * Started app 'mule-file-connector-example' * * Application plugins: * * - File Common Plugin : 1.3.2 * * - Sockets : 1.1.5 * * - File : 1.3.2 * * - HTTP : 1.5.11 * ********************************************************************** INFO 2021-08-10 18:49:59,058 [WrapperListener_start_runner] org.mule.runtime.core.internal.logging.LogUtil: ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + Mule is up and kicking (every 5000ms) + ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ INFO 2021-08-10 18:49:59,087 [WrapperListener_start_runner] org.eclipse.jetty.server.AbstractConnector: Started ServerConnector@1619078f{HTTP/1.1,[http/1.1]}{0.0.0.0:59321} INFO 2021-08-10 18:49:59,091 [WrapperListener_start_runner] org.mule.runtime.core.internal.logging.LogUtil: ********************************************************************** * - - + DOMAIN + - - * - - + STATUS + - - * ********************************************************************** * default * DEPLOYED * ********************************************************************** ******************************************************************************************************* * - - + APPLICATION + - - * - - + DOMAIN + - - * - - + STATUS + - - * ******************************************************************************************************* * mule-file-connector-example * default * DEPLOYED * ******************************************************************************************************* INFO 2021-08-10 22:11:11,528 [http.listener.01 SelectorRunner] org.mule.service.http.impl.service.util.DefaultRequestMatcherRegistry: No listener found for request: (GET)/read INFO 2021-08-10 22:11:11,564 [http.listener.01 SelectorRunner] org.mule.service.http.impl.service.util.DefaultRequestMatcherRegistry: Available listeners are: [([*])/delete/, ([*])/readfile/, ([*])/createdir/, ([*])/listfiles/, ([*])/move/, ([*])/copy/, ([*])/listdir/, ([*])/listall/, ([*])/listnonrecfiles/] INFO 2021-08-10 22:20:51,134 [[MuleRuntime].cpuIntensive.01: [mule-file-connector-example].list_files_and_directories_flow.CPU_INTENSIVE @676a3901] [event: 1e7629a0-f9fb-11eb-a30a-a44cc84ce10c] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: List all the files and folder information from a given directory done!! INFO 2021-08-10 22:26:48,168 [[MuleRuntime].cpuIntensive.03: [mule-file-connector-example].list_files_only_flow.CPU_INTENSIVE @6c98400b] [event: f36da9d1-f9fb-11eb-a30a-a44cc84ce10c] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: List all the files information (without folder) from a given directory done!! INFO 2021-08-10 22:36:01,158 [[MuleRuntime].cpuIntensive.02: [mule-file-connector-example].list_directory_only_flow.CPU_INTENSIVE @2a63315a] [event: 3d0f0a11-f9fd-11eb-a30a-a44cc84ce10c] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: List all the directories/folder information(without files) done!! INFO 2021-08-10 22:41:35,614 [[MuleRuntime].cpuIntensive.04: [mule-file-connector-example].list_files_non_recursive_flow.CPU_INTENSIVE @6d0a0cc8] [event: 046a3211-f9fe-11eb-a30a-a44cc84ce10c] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: List all the file or directories information irrespective of any subfolder from a given path done!!
For move file operation we need to drag and drop move oparation from mule palette to mule message flow as below, also we need to configure the directory for processed file.
<flow name="move_file_flow" doc:id="750ca921-e79f-4b60-b448-9b2801ae9df7"> <http:listener doc:name="Listener" doc:id="36277dc4-5454-4ed8-bd9c-0061d05b8ba0" config-ref="HTTP_Listener_config" path="/move" /> <file:move doc:name="Move" doc:id="f8df27a5-a3d9-45ba-baca-b8e507c6f61c" sourcePath="D:\development\AnypointStudio\studio-workspace\mule-file-connector-example\src\test\resources\data\input\processing\file2.txt" targetPath="D:\development\AnypointStudio\studio-workspace\mule-file-connector-example\src\test\resources\data\output" config-ref="File_Config_Resources"> <reconnect /> </file:move> </flow>
When we will call move file operation, file “file2.txt” will be moved to the “output” directory under data from “processing” directory as shown below.
Delete file operation will delete a file from directory permanantly.
<flow name="file_delete_flow" doc:id="e3712fc3-f3af-43ff-994f-145b3cda026f"> <http:listener doc:name="Listener" doc:id="e2dec67d-8efe-4152-8c23-016ad250a892" config-ref="HTTP_Listener_config" path="/delete" /> <file:delete doc:name="Delete" doc:id="903734d0-40e6-46f0-91f5-580250622488" config-ref="File_Config_Resources" path="D:\development\AnypointStudio\studio-workspace\mule-file-connector-example\src\test\resources\data\output" /> </flow>
Once we call the /delete operation, “output” folder in side data will be deleted with files as shown below.
In this mule flow we will create a new direcotory and will copy the file from source to destination.
<flow name="file_create_directory_flow" doc:id="dc7080f4-1ca6-4394-a3e9-f2f854d5937b"> <http:listener doc:name="Listener" doc:id="f44cbd58-3e93-4453-9436-60f2de3cc22d" config-ref="HTTP_Listener_config" path="/createdir" /> <file:create-directory doc:name="Create directory" doc:id="6ef6dab7-dce9-44b8-8dcd-c0eb915338dd" config-ref="File_Config_Resources" directoryPath="D:\development\AnypointStudio\studio-workspace\mule-file-connector-example\src\test\resources\data\failed" /> </flow> <flow name="file_copy_flow" doc:id="031391f5-d786-4212-8154-84fd59a41832"> <http:listener doc:name="Listener" doc:id="55177491-15e9-423d-946c-1b02797562d8" config-ref="HTTP_Listener_config" path="/copy" /> <file:copy doc:name="Copy" doc:id="6a825bc1-bfb5-41a5-ac82-d67ee69b083f" sourcePath="D:\development\AnypointStudio\studio-workspace\mule-file-connector-example\src\test\resources\data\input\processing\" targetPath="D:\development\AnypointStudio\studio-workspace\mule-file-connector-example\src\test\resources\data\failed" config-ref="File_Config_Resources" /> </flow>
When we will hit createdir REST API on broswer, a new empty directory will created with named “failed”.
Note: if directory already exist then it will throw exception “Directory ‘D:\development\AnypointStudio\studio-workspace\mule-file-connector-example\src\test\resources\data\failed’ already exist”, we can override by using configurations.
When we will hit copy operation, it will copy “processing” folder to “failed” directory as per flow and configuration.
File connector provides a listener that polls a directory for files that have been created/updated. A message is generated for each file that is found.
Let’s see how we trigger flow on new/updated file.
Drag and drop on New or Updated File from mule palette and configure the below parameters.
Configure the below parameters:
Directory: provide the directory path
Matcher: Use a matcher to filter the files
Watermark: Use the watermark parameter to only pick files that have been created or updated after the last poll was executed by default it is disabled
Schedule Frequency: configure the frequency to poll the directory
Once the file is processed you can configure below parameters
Auto Delete: file delete after processing default is false
Move to directory: configure the directory for processed file
Rename: rename file after processing
Thank you. Keep Learning. Have a great Day 🙂