Service with Ziti Desktop Edge for Mac (ERR_CONNECTION_TIMED_OUT)

How things work
1

Hi :wave:,

I’m totally new to NetFoundry and openZiti. So consider this as a noob question.

Setup

I tried to create a simple setup to make my first steps. It involves two endpoints, a service and an appWAN.

Samsung (Mobile Edge) → NF Hosted Edge Router → Macbook (Desktop Edge on Mac) → local webserver bound to 0.0.0.0:8080

Problem

When I try to access the service (http://macbook.nf:8080):

  • DNS resolves to 100.64.0.10:8080
  • curl reports:
    Failed to connect to macbook.nf port 8080 after 75013 ms: Operation timed out

The request seems to not reach the service, but I don’t know where it get lost.
I attached the config as JSON and the appex.log with trace loglevel.

Thanks,
Sven

endpoints.json

[
  {
    "id": "3371227c-ca0b-478c-a717-affd3d3d4ba0",
    "ownerIdentityId": "c0846161-319f-44e7-930e-0e8417237588",
    "createdBy": "c0846161-319f-44e7-930e-0e8417237588",
    "createdAt": "2022-07-18T17:41:14.307840Z",
    "updatedAt": "2022-07-19T09:07:10.652080Z",
    "deletedBy": null,
    "deletedAt": null,
    "networkId": "d04510e2-81d5-4cdf-91d5-b74839cbe6a8",
    "zitiId": "yhvmLeSkS2",
    "name": "MacBook",
    "typeId": "Device",
    "appId": "io.netfoundry.ZitiPacketTunnel.PacketTunnelProvider",
    "appVersion": "2.23 (457)",
    "branch": "HEAD",
    "revision": "bb20e9a",
    "type": "ziti-sdk-c",
    "version": "0.28.13",
    "arch": "arm64",
    "os": "Darwin",
    "osRelease": "21.4.0",
    "osVersion": "Darwin Kernel Version 21.4.0: Fri Mar 18 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000",
    "hasApiSession": false,
    "hasEdgeRouterConnection": false,
    "lastOnlineAt": "2022-07-19T08:35:34Z",
    "syncId": null,
    "syncResourceId": null,
    "attributes": [
      "#all"
    ],
    "mfaEnabled": true,
    "jwt": null,
    "online": false,
    "jwtExpiresAt": null
  },
  {
    "id": "ba647953-d684-41d9-8d6f-0b901b3a25c4",
    "ownerIdentityId": "c0846161-319f-44e7-930e-0e8417237588",
    "createdBy": "c0846161-319f-44e7-930e-0e8417237588",
    "createdAt": "2022-07-17T18:54:35.531176Z",
    "updatedAt": "2022-07-19T08:56:13.851639Z",
    "deletedBy": null,
    "deletedAt": null,
    "networkId": "d04510e2-81d5-4cdf-91d5-b74839cbe6a8",
    "zitiId": "fOGduxSkS",
    "name": "Samsung",
    "typeId": "Device",
    "appId": "org.openziti.mobile",
    "appVersion": "v0.7.9",
    "branch": null,
    "revision": "ffbf119",
    "type": "ziti-sdk-java",
    "version": "0.23.11",
    "arch": "aarch64",
    "os": "Android",
    "osRelease": "12",
    "osVersion": "2022-06-01",
    "hasApiSession": true,
    "hasEdgeRouterConnection": true,
    "lastOnlineAt": "2022-07-19T08:56:14Z",
    "syncId": null,
    "syncResourceId": null,
    "attributes": [
      "#all"
    ],
    "mfaEnabled": false,
    "jwt": null,
    "online": true,
    "jwtExpiresAt": null
  }
]

services.json

[
  {
    "id": "1cd88136-9b4b-4cb7-9763-63bc91f9882a",
    "networkId": "d04510e2-81d5-4cdf-91d5-b74839cbe6a8",
    "zitiId": "hzk1R3vkv2",
    "name": "hello",
    "encryptionRequired": true,
    "modelType": "TunnelerToSdk",
    "ownerIdentityId": "c0846161-319f-44e7-930e-0e8417237588",
    "createdBy": "c0846161-319f-44e7-930e-0e8417237588",
    "createdAt": "2022-07-19T08:05:18.314595Z",
    "updatedAt": "2022-07-19T08:05:19.931005Z",
    "deletedBy": null,
    "deletedAt": null,
    "configIdByConfigTypeId": {
      "725e02ab-2bda-4867-ac99-036bfb3ecd08": "ab01732b-2f1f-40eb-bfb3-40ffa3b5eb03"
    },
    "attributes": [
      "#hello"
    ],
    "model": {
      "clientIngress": {
        "host": "macbook.nf",
        "port": 8080
      },
      "bindEndpointAttributes": [
        "@MacBook"
      ],
      "edgeRouterAttributes": []
    }
  }
]

appwans.json

[
  {
    "id": "69765b00-994d-4d14-8f39-002e5c7f2ed8",
    "ownerIdentityId": "c0846161-319f-44e7-930e-0e8417237588",
    "createdBy": "c0846161-319f-44e7-930e-0e8417237588",
    "createdAt": "2022-07-18T15:54:33.315612Z",
    "updatedAt": "2022-07-18T15:54:33.914084Z",
    "deletedBy": null,
    "deletedAt": null,
    "name": "hello",
    "zitiId": "iomP0Zv9v2",
    "networkId": "d04510e2-81d5-4cdf-91d5-b74839cbe6a8",
    "serviceAttributes": [
      "#hello"
    ],
    "endpointAttributes": [
      "#all"
    ],
    "postureCheckAttributes": [
      "@zitimfa"
    ]
  }
]

appex.log

[2022-07-19T09:25:03.705Z]   TRACE tunnel-sdk:netif_shim.c:34 netif_shim_output() 
writing packet TCP[100.64.0.10:8080 -> 100.64.0.1:54116] len=40
[2022-07-19T09:25:03.705Z]   TRACE tunnel-sdk:netif_shim.c:34 netif_shim_output() writing packet TCP[100.64.0.10:8080 -> 100.64.0.1:54113] len=40
[2022-07-19T09:25:12.462Z] VERBOSE ziti-sdk:posture.c:184 ziti_send_posture_data() ztx[0] starting to send posture data
[2022-07-19T09:25:12.462Z]   DEBUG ziti-sdk:posture.c:197 ziti_send_posture_data() ztx[0] posture checks must_send set to TRUE, new_session_id[FALSE], must_send_every_time[TRUE], new_controller_instance[FALSE]
[2022-07-19T09:25:12.462Z] VERBOSE ziti-sdk:posture.c:222 ziti_send_posture_data() ztx[0] checking posture queries on 1 service(s)
[2022-07-19T09:25:12.462Z]   DEBUG ziti-sdk:posture.c:512 ziti_pr_send_bulk() ztx[0] no change in posture data, not sending
[2022-07-19T09:25:12.823Z]   DEBUG ziti-sdk:ziti_ctrl.c:131 start_request() ctrl[9949d440-a679-4f0e-b269-5b7a9af77894.production.netfoundry.io] starting GET[/current-api-session/service-updates]
[2022-07-19T09:25:12.859Z]   DEBUG ziti-sdk:ziti_ctrl.c:166 ctrl_resp_cb() ctrl[9949d440-a679-4f0e-b269-5b7a9af77894.production.netfoundry.io] received headers GET[/current-api-session/service-updates]
[2022-07-19T09:25:12.859Z]   DEBUG ziti-sdk:ziti_ctrl.c:322 ctrl_body_cb() ctrl[9949d440-a679-4f0e-b269-5b7a9af77894.production.netfoundry.io] completed GET[/current-api-session/service-updates] in 0.035 s
[2022-07-19T09:25:12.859Z] VERBOSE ziti-sdk:ziti.c:1137 check_service_update() ztx[0] not updating: last_update is same previous (2022-07-19T09:24:12.466Z == 2022-07-19T09:24:12.466Z)
[2022-07-19T09:25:12.859Z] VERBOSE ziti-sdk:ziti.c:1174 ziti_services_refresh() ztx[0] scheduling service refresh 15 seconds from now
[2022-07-19T09:25:12.975Z]   TRACE ziti-sdk:channel.c:383 ziti_channel_send_for_reply() ch[1] => ct[0003] seq[3] len[0]
[2022-07-19T09:25:12.975Z]   TRACE ziti-sdk:channel.c:788 on_write() on_write(0x133410d60,0)
[2022-07-19T09:25:12.997Z]   TRACE ziti-sdk:channel.c:383 ziti_channel_send_for_reply() ch[0] => ct[0003] seq[1] len[0]
[2022-07-19T09:25:12.997Z]   TRACE ziti-sdk:channel.c:788 on_write() on_write(0x133410d60,0)
[2022-07-19T09:25:13.018Z]   TRACE ziti-sdk:channel.c:851 on_channel_data() ch[1] on_data [len=57]
[2022-07-19T09:25:13.018Z]   TRACE ziti-sdk:channel.c:539 process_inbound() ch[1] <= ct[0002] seq[3] len[0] hdrs[37]
[2022-07-19T09:25:13.018Z]   TRACE ziti-sdk:channel.c:549 process_inbound() ch[1] completing msg seq[3] body+hrds=0+37, in_offset=0, want=37, got=37
[2022-07-19T09:25:13.018Z]   TRACE ziti-sdk:channel.c:560 process_inbound() ch[1] message is complete seq[3] ct[0002]
[2022-07-19T09:25:13.018Z] VERBOSE ziti-sdk:channel.c:585 latency_reply_cb() ch[1] latency is now 48
[2022-07-19T09:25:13.018Z]   TRACE ziti-sdk:channel.c:851 on_channel_data() ch[0] on_data [len=57]
[2022-07-19T09:25:13.018Z]   TRACE ziti-sdk:channel.c:539 process_inbound() ch[0] <= ct[0002] seq[1] len[0] hdrs[37]
[2022-07-19T09:25:13.018Z]   TRACE ziti-sdk:channel.c:549 process_inbound() ch[0] completing msg seq[1] body+hrds=0+37, in_offset=0, want=37, got=37
[2022-07-19T09:25:13.018Z]   TRACE ziti-sdk:channel.c:560 process_inbound() ch[0] message is complete seq[1] ct[0002]
[2022-07-19T09:25:13.018Z] VERBOSE ziti-sdk:channel.c:585 latency_reply_cb() ch[0] latency is now 20
[2022-07-19T09:25:24.990Z]   TRACE tunnel-sdk:tunnel_tcp.c:325 recv_tcp() received segment 100.64.0.1:54113->100.64.0.10:8080
[2022-07-19T09:25:24.990Z] VERBOSE tunnel-sdk:tunnel_tcp.c:346 recv_tcp() received SYN on active connection: client=tcp:100.64.0.1:54113, service=hello
[2022-07-19T09:25:24.990Z]   TRACE tunnel-sdk:netif_shim.c:34 netif_shim_output() writing packet TCP[100.64.0.10:8080 -> 100.64.0.1:54113] len=40
[2022-07-19T09:25:25.230Z]   TRACE tunnel-sdk:tunnel_tcp.c:325 recv_tcp() received segment 100.64.0.1:54116->100.64.0.10:8080
[2022-07-19T09:25:25.230Z] VERBOSE tunnel-sdk:tunnel_tcp.c:346 recv_tcp() received SYN on active connection: client=tcp:100.64.0.1:54116, service=hello
[2022-07-19T09:25:25.230Z]   TRACE tunnel-sdk:netif_shim.c:34 netif_shim_output() writing packet TCP[100.64.0.10:8080 -> 100.64.0.1:54116] len=40
[2022-07-19T09:25:27.860Z]   DEBUG ziti-sdk:ziti_ctrl.c:131 start_request() ctrl[9949d440-a679-4f0e-b269-5b7a9af77894.production.netfoundry.io] starting GET[/current-api-session/service-updates]
[2022-07-19T09:25:27.932Z]   DEBUG ziti-sdk:ziti_ctrl.c:166 ctrl_resp_cb() ctrl[9949d440-a679-4f0e-b269-5b7a9af77894.production.netfoundry.io] received headers GET[/current-api-session/service-updates]
[2022-07-19T09:25:27.932Z]   DEBUG ziti-sdk:ziti_ctrl.c:322 ctrl_body_cb() ctrl[9949d440-a679-4f0e-b269-5b7a9af77894.production.netfoundry.io] completed GET[/current-api-session/service-updates] in 0.071 s
[2022-07-19T09:25:27.932Z] VERBOSE ziti-sdk:ziti.c:1137 check_service_update() ztx[0] not updating: last_update is same previous (2022-07-19T09:24:12.466Z == 2022-07-19T09:24:12.466Z)
[2022-07-19T09:25:27.932Z] VERBOSE ziti-sdk:ziti.c:1174 ziti_services_refresh() ztx[0] scheduling service refresh 15 seconds from now
2

Was the test run from the Samsung towards the Macbook, and the appex.log is the Macbook?? I’ve not seen SYN received on active connection before on the server side, only on the client/ingress side when a connection isn’t completing, but that may not mean anything, could just be I haven’t run across it.
The message writing packet TCP[100.64.0.10:8080 → 100.64.0.1:54113] indicates a packet from the server port to the client, so it is making it through to the server, but the endpoint isn’t acknowledging the SYN/ACK with an ACK. Then, the client is resending a SYN. This happens on at least 2 connections. 54113 and 54116.
What are the logs like on the Samsung? The Feedback option in the client will create a support bundle, and you can email it to yourself to get the logs and some additional information easily.

3

hi @0xe282b0 Sven - I see the service config on your network. The service has to be “endpoint” hosted since you are hosting it on your device endpoint. Try deleting and recreating the service as endpoint hosted.

image
image1189×670 58.9 KB

image

4

Thanks it’s working!

@mike.gorman, yes there was never an ACK from the server. Good to know about support bundles.

@surendran.naidu, good catch. That was the cause. I overlooked it since the toggle is disabled after service creation and the service appeared as active in Ziti Desktop.

I already love this community!

2 Likes