Edit on GitHub

Mobile End-to-End (E2E) tests at Mattermost

This page describes how to write and run End-to-End (E2E) testing for Mobile Apps for both iOS and Android. Mobile products use Detox, which is a “gray box end-to-end testing and automation library for mobile apps.” See its documentation to learn more.

File Structure 

The folder structure is as follows:

|-- detox
  |-- e2e
    |-- support
    |-- test
    |-- config.json
    |-- environment.js
    |-- init.js
  |-- .babelrc
  |-- .detoxrc.json
  |-- package-lock.json
  |-- package.json

• /detox/e2e/support is the support folder, which is a place to put reusable behavior such as Server API and UI commands, or global overrides that should be available to all test files.
• /detox/e2e/test: To start writing tests, create a new file (e.g. login.e2e.js) in the /detox/e2e/test folder.
  • The subfolder naming convention depends on the test grouping, which is usually based on the general functional area (e.g. /detox/e2e/test/messaging/ for “Messaging”).
  • Test cases that require an Enterprise license should fall under /detox/e2e/test/enterprise/. This is to easily identify license requirements, both during local development and production testing for Enterprise features.
• /detox/.detoxrc.json: for Detox configuration.
• /detox/package.json : for all dependencies related to Detox end-to-end testing.

Writing an E2E Test 

This process has many similarities to writing an E2E test for the mattermost-webapp project. Before writing a script, ensure that it has a corresponding test case in Zephyr. All test cases may be found in this link. If test case is not available, feel free to prompt the QA team who will either search from an existing Zephyr entry or if it’s a new one, it will be created for you.

1. Create a test file based on the file structure aforementioned above.

2. Include Zephyr identification (ID) and title in the test description, following the format of it('[Zephyr_id] [title]') or it('[Zephyr_id]_[step] [title]') if the test case has multiple steps. For test case “MM-T109 RN apps: User can't send the same message repeatedly”, it should be:

   describe('Messaging', () => {
       it('MM-T109 User can\'t send the same message repeatedly', () => {
           // Test steps and assertion here
       }
   }
   

3. Target an element using available matchers. For best results, it is recommended to match elements by unique identifiers using testID. The identifier should follow the following format to avoid duplication, <location>.<modifier>.<element>.<identifier>, where:

   • location - can be a parent component, a main section, or a UI screen.
   • modifier - adds meaning to the element.
   • element - common terms like button, text_input, image, and the like.
   • identifier - could be unique ID of a post, channel, team or user, or a number to represent order.

4. Prefix each comment line with appropriate indicator. Each line in a multi-line comment should be prefixed accordingly. Separate and group test step comments and assertion comments for better readability.

   • # indicates a test step (e.g. // # Go to a screen)
   • * indicates an assertion (e.g. // * Check the title)

5. Simulate user interaction using available actions, and verify user interface (UI) expectations using expect. When using action, match, or another API specific to a particular platform, verify that the equivalent logic is applied so that the API does not impact other platforms. Always run tests in applicable platforms.

Running E2E Tests 

Testing Android Locally 

Local setup 

1. Install the latest Android SDK:

   sdkmanager "system-images;android-30;google_apis;x86"
   sdkmanager --licenses
   

2. Create the emulator using npm run e2e:android-create-emulator from the /detox folder. Android testing requires an emulator named detox_pixel_4_xl_api_30 and the script helps to create it automatically.

Complete a test run in debug mode 

This is the typical flow for local development and test writing:

1. Open a terminal window and run react-native packager by npm install && npm start from the root folder.
2. Open a second terminal window and:
   • Change directory to /detox folder.
   • Install npm packages by npm install.
   • Build the app together with the android test using npm run e2e:android-build.
   • Run the test using npm run e2e:android-test.
   • For running a single test, follow this example command: npm run e2e:android-test -- connect_to_server.e2e.ts.

Complete a test run in release mode 

This is the typical flow for CI test run:

1. Build a release app by running npm install && npm run e2e:android-build-release from the /detox folder.
2. Run a test using npm run e2e:android-test-release from the /detox folder.

Testing iOS Locally 

Local setup 

1. Install applesimutils:

   brew tap wix/brew
   brew install applesimutils
   

2. Set XCode’s build location so that the built app, especially debug, is expected at the project’s location instead of the Library’s folder which is unique/hashed.
3. Open XCode, then go to XCode > Preferences > Locations.
4. Under Derived Data, click Advanced….
5. Select Custom > Relative to Workspace, then set Products as Build/Products.
6. Click Done to save the changes.

Complete a test run in debug mode 

1. In one terminal window, run npm run ios from the root folder.
2. In another terminal window, run npm i then npm run e2e:ios-test from the /detox folder.
   • For running a single test, follow this example command: npm run e2e:ios-test -- connect_to_server.e2e.ts.

Complete a test run in release mode 

1. Build the release app by running npm run build:ios-sim from the root folder or npm run e2e:ios-build-release from within the /detox folder.
2. Run the test using npm run e2e:ios-test-release from the /detox folder.

Environment variables 

Test configurations are defined at test_config.js and environment variables are used to override default values. In most cases you don’t need to change the values, because it makes use of the default local developer setup. If you do need to make changes, you may override by exporting, e.g. export SITE_URL=<site_url>.

Variable	Description
SITE_URL	Host of test server. Default: http://localhost:8065 for iOS or http://10.0.2.2:8065 for Android.
ADMIN_USERNAME	Admin’s username for the test server. Default: sysadmin when server is seeded by make test-data.
ADMIN_PASSWORD	Admin’s password for the test server. Default: Sys@dmin-sample1 when server is seeded by make test-data.
LDAP_SERVER	Host of LDAP server. Default: localhost
LDAP_PORT	Port of LDAP server. Default: 389