[stratum-dev] Update Bazel version to 2.2.0

Deval Bhamare Deval.Bhamare at kau.se
Thu Apr 16 12:50:39 UTC 2020


Hello Yi and team,


With reference to your email below:


The current way to support Stratum + BSP is to build the SDE, BSP, and Stratum on the switch


>> Does it mean, pre-built Stratum container cannot be used with BSP? Then what is the meaning of below flag in the container start script?


ONLP_ARG="--env WITH_ONLP=false \ -
-env PLATFORM=x86-64-accton-wedge100bf-32x-r0"




Isn't the above option mean stratum will use BSP?  If not, then with ONLP false and without BSP, how will switch work? Is it by reading the JSON for the specified platform above (x86-64-accton-wedge100bf-32x-r0)?


can you please explain?


Note: We still are facing errors when we tr yo build stratum with BSP.


Thanks,


Deval.

________________________________
From: Yi Tseng <yi at opennetworking.org>
Sent: Thursday, April 2, 2020 2:02:40 PM
To: Deval Bhamare
Cc: max at opennetworking.org; stratum-dev at lists.stratumproject.org
Subject: Re: [stratum-dev] Update Bazel version to 2.2.0

Hi Deval,

The Stratum container image does not include the BSP library from each vendor since the origin goal is to use a general open-source platform library from ONL.

The current way to support Stratum + BSP is to build the SDE, BSP, and Stratum on the switch (you can also build it on the server and copy every necessary libraries and files to the switch). Also, I believed that you can build everything(SDE, BSP, Stratum) on a Ubuntu-based switch and run it.

I've tested to build SDE, BSP, and Stratum on out BF2556X and I can confirm that ONOS can connect to the Stratum.
Currently, we are using ONL-based OS in our lab. I can try re-installed the switch to Ubuntu-based once I can access the hardware.

Yi


On Mon, Mar 30, 2020 at 8:25 PM Deval Bhamare <Deval.Bhamare at kau.se<mailto:Deval.Bhamare at kau.se>> wrote:

Hello again,


So as per my understanding, when we use pre-built docker version of the Stratum, it doesn't refer to local locally built BSP and BF-SDE. Instead it uses its own pre-build BF-SDE and BSP. Is it correct?


In that case, with reference to the email from Stordis-support below, I would like to confirm that below point have been taken care of while building the updated docker image.


Stordis_support message:


"It is important that you build the BSP with the --with-tof-brgup-plat option that you can also pass through the p4studio_build script (see installation guide). Be sure to spell it correctly, or else p4studio_build will silently ignore the option and build the BSP incorrectly… (there will be just a log entry in the installation log saying that the option was unknown)

Note too that there was a problem in the BSP zip file that is available on our FTP server – the bf-platforms.tgz in the packages directory was actually the reference BSP version. If you had used this tgz (instead of the bf-platforms directory that is present in the directory above), the BSP would have been built incorrectly too. In the meantime I have uploaded a corrected version of the zip file to our FTP server."



Can you please confirm?


Thanks,


Deval.

________________________________
From: Yi Tseng <yi at opennetworking.org<mailto:yi at opennetworking.org>>
Sent: Monday, March 30, 2020 7:32:05 AM
To: Deval Bhamare
Cc: max at opennetworking.org<mailto:max at opennetworking.org>; stratum-dev at lists.stratumproject.org<mailto:stratum-dev at lists.stratumproject.org>
Subject: Re: [stratum-dev] Update Bazel version to 2.2.0

Hi Deval,

We've updated the container image and the script to start the container:
https://github.com/stratum/stratum/blob/master/stratum/hal/bin/barefoot/docker/start-stratum-container.sh

You can now try the following command to start the Stratum container with no ONLP support:
PLATFORM=x86-64-stordis-bf2556x-1t-r0 SDE_VERSION=8.9.2 ./start-stratum-container.sh

For the "unimplemented" message you saw, the SDE 8.9.2 uses old version PI library which doesn't implement some features (reading default entry from a table).
You can ignore that warning or use a newer version of Stratum (with SDE >= 9.0.0)

Yi


On Sun, Mar 29, 2020 at 8:31 PM Deval Bhamare <Deval.Bhamare at kau.se<mailto:Deval.Bhamare at kau.se>> wrote:

Hello again,



Just don't want to overload with the information, but I wanted to mention that, with the pre-built docker image option, I had a look a the ./start-stratum-container.sh


Since I want to run Stratum with BSP, I made a change as:


ONLP_ARG="--env WITH_ONLP=false"


Then I started the container 8.9.2-3.16.56-OpenNetworkLinux and it executed the Stratum successfully. Stratum open bf-sde prompt.


Also ss ouptput is:


State      Recv-Q Send-Q          Local Address:Port                         Peer Address:Port

LISTEN     0      128                         *:28000                                   *:*                   users:(("stratum_bf_no_o",pid=10,fd=36))

LISTEN     0      128                 127.0.0.1:28000<http://127.0.0.1:28000>                                   *:*                   users:(("stratum_bf_no_o",pid=10,fd=35))

LISTEN     0      128                 127.0.0.1:28003<http://127.0.0.1:28003>                                   *:*                   users:(("stratum_bf_no_o",pid=10,fd=32))

LISTEN     0      5                           *:9999                                    *:*                   users:(("stratum_bf_no_o",pid=10,fd=27))


I am getting following messages now:


 ONOS-CLI logs for the confirmation below:


tofino at root > devices                                                                                 22:02:35

id=device:tofino, available=true, local-status=connected 4m52s ago, role=MASTER, type=SWITCH, mfr=Barefoot Networks, hw=Tofino, sw=Stratum, serial=unknown, chassis=0, driver=stratum-tofino:vepg-pipeconf, locType=none, managementAddress=grpc://127.0.0.1:28000?device_id=1<http://127.0.0.1:28000?device_id=1>, name=device:tofino, p4DeviceId=1, protocol=P4Runtime, gNMI, gNOI


Logs on stratum bf-sde are:


bf-sde.lld> E0329 22:01:52.649449   249 PI-device_mgr.cpp:0] Reading default entry not supported yet

E0329 22:01:52.649509   249 pi_node.cc:187] Return Error: toUtilStatus(status, details) failed with generic::unimplemented:

E0329 22:01:52.649549   249 p4_service.cc:307] Failed to read forwarding entries from node 1:


ONOS Logs:


2020-03-29T22:08:22,649 | WARN  | grpc-default-executor-4 | P4RuntimeClientImpl              | 230 - org.onosproject.onos-protocols-grpc-ctl - 2.4.0.SNAPSHOT | Error while performing READ on device:tofino: UNIMPLEMENTED



So, now ONOS can talk to Tofino through Stratum, but there are a few errors. First of all, is it OK to explicitly use this option: "--env WITH_ONLP=false" If yes, am I missing some steps here?  Any suggestion?



(Note: If I use default value of ONLP_ARG as per the original script, I get error as:


Use default flag file

Cannot find /usr/local/lib/modules/bf_kdrv.ko, skip installing the Kernel module

/usr/local/bin/stratum_bf: symbol lookup error: /usr/local/bin/stratum_bf: undefined symbol: onlp_sw_init

)



Thank you,


Deval.


________________________________
From: Deval Bhamare
Sent: Sunday, March 29, 2020 12:20:10 PM
To: Maximilian Pudelko
Cc: Yi Tseng; stratum-dev at lists.stratumproject.org<mailto:stratum-dev at lists.stratumproject.org>
Subject: Re: [stratum-dev] Update Bazel version to 2.2.0


Hello Maximilian,


There is a littler confusion here. I am not using pre-built containerized version of Stratum.

However, I am locally building it by starting a development container with command "./setup_dev_env.sh  --pull -- -p 28000:28000" (I have mentioned it in the earlier image as well) as mentioned here: https://github.com/stratum/stratum

and then I try to build and run stratum as mentioned here: https://github.com/stratum/stratum/blob/master/stratum/hal/bin/barefoot/README.md



Now with the build-from-scratch using development container approach, I followed your steps, however, when I run ss -tnlp from the container (afterdocker exec -ti bash), I get below and listen on  127.0.0.1:28000<http://127.0.0.1:28000> is missing. I get only below.



tofino at 39e19c57ea27:/stratum$ ss -tlnp

State       Recv-Q Send-Q     Local Address:Port                    Peer Address:Port

LISTEN      0      128            127.0.0.1:56929<http://127.0.0.1:56929>                              *:*                   users:(("java",pid=21,fd=110))

LISTEN      0      1                      *:8008                               *:*                   users:(("stratum_bf",pid=310,fd=26))

Command to build Stratum is: bazel build //stratum/hal/bin/barefoot:stratum_bf --define phal_with_onlp=false --define sde_ver=8.9.2

Command to run Stratum is:

     ./bazel-bin/stratum/hal/bin/barefoot/stratum_bf \
       --external_stratum_urls=0.0.0.0:28000<http://0.0.0.0:28000> \
       --grpc_max_recv_msg_size=256 \
       --bf_sde_install=$BF_SDE_INSTALL \
       --persistent_config_dir=/stratum/config/ \
       --forwarding_pipeline_configs_file=/stratum/config/p4_pipeline.pb.txt \
       --chassis_config_file=/stratum/config/chassis_config.pb.txt \
       --write_req_log_file=/stratum/config/p4_writes.pb.txt \
       --bf_sim




[Note: I had also tried the option of using pre-built docker container, with 8.9.1-3.16.56-OpenNetworkLinux, however, in this case I get error:

Cannot find /usr/local/share/stratum/x86-64-stordis-bf2556x-1t-r0.json

I had reported that also and tried new container but didn't work]


Thanks,

Deval.


________________________________
From: Maximilian Pudelko <max at opennetworking.org<mailto:max at opennetworking.org>>
Sent: Sunday, March 29, 2020 1:46:04 AM
To: Deval Bhamare
Cc: Yi Tseng; stratum-dev at lists.stratumproject.org<mailto:stratum-dev at lists.stratumproject.org>; Andreas Kassler
Subject: Re: [stratum-dev] Update Bazel version to 2.2.0

Hi Deval,

There are many things going on in your setup, let's do this step by step:

1. Port 8008 is NOT what you want for ONOS. I'm no Tofino expert, but this seems to be a separate endpoint for something else (debugging?). We always use port 28000.

2. 2020-03-28 13:06:53.584190 BF_PLTFM ERROR - pltfm_mgr: platform init failed  #<<< CAN IT BE THE ISSUE?
Depends. It's probably not good, but as long as you see this line in the Stratum logs, ONOS should be able to connect:
E0328 23:08:15.903065    13 hal.cc:234] Stratum external facing services are listening to 0.0.0.0:28000<http://0.0.0.0:28000>, localhost:28000...
Judging from the missing listening port, I'd say this could be the issue.

3. If you run ONOS in a container too, they have to reside in the same network and you have to figure out the docker-internal IP for the Stratum container:
# docker network ls
NETWORK ID          NAME                DRIVER              SCOPE
e737dd0dd90f        bridge              bridge              local
d2783152be24        host                host                local
7028e832e130        none                null                local

# docker network inspect bridge
[...]
"Containers": {
            "518c901167919d34205959f1105aedebbb99a9452d191409ccf71a3908e14e56": {
                "Name": "zen_jennings",
                "EndpointID": "a52025a671c9968372c52c7ed6b61df5a3168493466bd4b831c4728089b77163",
                "MacAddress": "02:42:ac:11:00:02",
                "IPv4Address": "172.17.0.2/16",
                "IPv6Address": ""
            }
        },
[...]

You could also try running them in the host network (--network=host), that should make them behave like native processes with fully exposed networking.

4. I used a wrong switch in my previous post, here's how it looks on a Tofino:

  *    ./start-stratum-container.sh (let it run and open new terminal tab)
  *   docker container ls
CONTAINER ID        IMAGE                                                      COMMAND                  CREATED             STATUS              PORTS                      NAMES
518c90116791        stratumproject/stratum-bf:9.1.0-4.14.49-OpenNetworkLinux   "/bin/sh -c /stratum…"   12 minutes ago      Up 12 minutes       0.0.0.0:28000->28000/tcp   zen_jennings
  *   ss -ltnp
LISTEN      0      128                    :::28000                              :::*                   users:(("docker-proxy",pid=1179,fd=4))
  *   docker exec -ti 518c90116791 bash (switch into the Stratum container)
  *   apt update && apt install iproute
  *   ss -tlnp
State       Recv-Q Send-Q      Local Address:Port                     Peer Address:Port
LISTEN      0      5                       *:9999                                *:*                   users:(("stratum_bf",pid=13,fd=27))
LISTEN      0      128                     *:28000                               *:*                   users:(("stratum_bf",pid=13,fd=40))
LISTEN      0      128             127.0.0.1:28000<http://127.0.0.1:28000>                               *:*                   users:(("stratum_bf",pid=13,fd=39))
LISTEN      0      1                       *:9001                                *:*                   users:(("stratum_bf",pid=13,fd=28))
  *   exit (leave container)
  *   docker run -ti debian:9 bash (Start new container to simulate ONOS)
  *   apt update && apt install netcat
  *   nc -v 172.17.0.2 28000
172.17.0.2<http://172.17.0.2>: inverse host lookup failed: Unknown host
(UNKNOWN) [172.17.0.2] 28000 (?) open

For reference, the used versions:

# docker images
REPOSITORY                  TAG                              IMAGE ID            CREATED             SIZE
stratumproject/stratum-bf   9.1.0-4.14.49-OpenNetworkLinux   e4615d64018f        2 weeks ago         214MB

For reference, these are the versions we use:
# docker version
Client: Docker Engine - Community
 Version:           19.03.2
 API version:       1.40
 Go version:        go1.12.8
 Git commit:        6a30dfca03
 Built:             Thu Aug 29 05:29:49 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.2
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.8
  Git commit:       6a30dfca03
  Built:            Thu Aug 29 05:28:23 2019
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.6
  GitCommit:        894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc:
  Version:          1.0.0-rc8
  GitCommit:        425e105d5a03fabd737a126ad93d62a9eeede87f
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

# uname -a
Linux Stordis 4.14.49-OpenNetworkLinux #1 SMP Thu Jul 18 08:52:25 UTC 2019 x86_64 GNU/Linux


Max

On Sat, Mar 28, 2020 at 7:31 AM Deval Bhamare <Deval.Bhamare at kau.se<mailto:Deval.Bhamare at kau.se>> wrote:

Hello again,


Since I could not any port activity on 28000 inside stratum and stratum logs shown only listen activity on 8008, I used following command to do explicitly map the IP/ports to host:


./setup_dev_env.sh  --pull --  -p 192.168.60.131:28008<http://192.168.60.131:28008>:8008


Then I tried to push the netcfg.json with following URL and this time the communication channel is established. Logs from ONOS and stratum are below (BOLD RED).


URL:


"managementAddress": "grpc://192.168.60.131:28008?device_id=0<http://192.168.60.131:28008?device_id=0>",


(I tired both deviceID 0 and 1, but it must be 0 as bf_shell and stratum Log both confirm: )



ONOS:


2020-03-28T13:49:46,158 | WARN  | onos-gdp-0       | PiPipeconfManager                | 193 - org.onosproject.onos-core-net - 2.4.0.SNAPSHOT | Ignoring PortStatisticsDiscovery behaviour from pipeconf vepg-pipeconf, but using the one provided by stratum-tofino driver...

2020-03-28T13:49:46,184 | INFO  | onos-gdp-0       | GrpcChannelControllerImpl        | 230 - org.onosproject.onos-protocols-grpc-ctl - 2.4.0.SNAPSHOT | Creating new gRPC channel grpc://192.168.60.131:28008?device_id=0.<http://192.168.60.131:28008?device_id=0.>.. << #THERE IS SUCCESS HERE THIS TIME AS COMPARED TO PREVIOUS TIME


Stratum:


Tcl server: connection accepted on port 8008

Partial recv: 30 of 40 so far..  <<#STRATUM CONFIRMS IT



However, I am not sure, it is the correct entry point? The reason is ONOS still cannot find the device:tofino (highlighted Yellow RED)


2020-03-28T13:49:46,306 | INFO  | onos-gdp-0       | DeviceManager                    | 193 - org.onosproject.onos-core-net - 2.4.0.SNAPSHOT | Local role is MASTER for device:tofino

2020-03-28T13:49:46,311 | WARN  | onos-event-dispatch-default0 | ModelCache                       | 211 - org.onosproject._onos-gui2-base-jar - 2.4.0.SNAPSHOT | DeviceID device:tofino not found as a UiDevice

2020-03-28T13:49:46,313 | INFO  | onos-gdp-0       | GeneralDeviceProvider            | 235 - org.onosproject.onos-providers-general-device - 2.4.0.SNAPSHOT | Notifying role MASTER (preference 0) for term 1 to device:tofino

2020-03-28T13:49:46,315 | WARN  | onos-event-dispatch-default0 | ModelCache                       | 211 - org.onosproject._onos-gui2-base-jar - 2.4.0.SNAPSHOT | DeviceID device:tofino not found as a UiDevice

2020-03-28T13:49:46,316 | WARN  | onos-device-manager-background | DeviceManager                    | 193 - org.onosproject.onos-core-net - 2.4.0.SNAPSHOT | Node was instructed to be MASTER role for device:tofino, but this node cannot reach the device.  Relinquishing role.    <<#STILL NO DEVICE FOUND. I TRIED BOTH ID 0 and 1 in the URL

2020-03-28T13:49:46,317 | INFO  | onos-gdp-0       | StreamClientImpl                 | 237 - org.onosproject.onos-protocols-p4runtime-ctl - 2.4.0.SNAPSHOT | Setting mastership on device:tofino... master=true, newElectionId=20, masterElectionId=null, sessionOpen=false

2020-03-28T13:49:46,320 | WARN  | onos-event-dispatch-default0 | ModelCache                       | 211 - org.onosproject._onos-gui2-base-jar - 2.4.0.SNAPSHOT | DeviceID device:tofino not found as a UiDevice

2020-03-28T13:49:46,321 | WARN  | onos-event-dispatch-default0 | ModelCache                       | 211 - org.onosproject._onos-gui2-base-jar - 2.4.0.SNAPSHOT | DeviceID device:tofino not found as a UiDevice

2020-03-28T13:49:46,321 | WARN  | onos-event-dispatch-default0 | ModelCache                       | 211 - org.onosproject._onos-gui2-base-jar - 2.4.0.SNAPSHOT | DeviceID device:tofino not found as a UiDevice

2020-03-28T13:49:46,336 | INFO  | onos-topo-build-1 | TopologyManager                  | 193 - org.onosproject.onos-core-net - 2.4.0.SNAPSHOT | Topology DefaultTopology{time=12233603415182, creationTime=1585403386336, computeCost=124274, clusters=0, devices=0, links=0} changed

2020-03-28T13:49:46,365 | INFO  | onos-gdp-0       | DeviceManager                    | 193 - org.onosproject.onos-core-net - 2.4.0.SNAPSHOT | Device device:tofino registered




Please confirm, is it in the right direction or port 8008 is completely different? What can be the reason for device:tofino not able to find?


Thank you,

Deval.


________________________________
From: stratum-dev <stratum-dev-bounces at lists.stratumproject.org<mailto:stratum-dev-bounces at lists.stratumproject.org>> on behalf of Deval Bhamare via stratum-dev <stratum-dev at lists.stratumproject.org<mailto:stratum-dev at lists.stratumproject.org>>
Sent: Saturday, March 28, 2020 3:22:16 PM
To: Maximilian Pudelko
Cc: stratum-dev at lists.stratumproject.org<mailto:stratum-dev at lists.stratumproject.org>
Subject: Re: [stratum-dev] Update Bazel version to 2.2.0


Hello Maximilian,


Thank you for your reply and the useful suggestions. So as a first step I executed on switch ss -tnlp and below is the output. Surprisingly, there is no Stratum anywhere there, however statrum is also running in a container and the short logs are displayed below. I also executed ss command inside the container and those are also pasted below, if that may help.


ss -tnlp outputs in different times are as follows:


- from tofino, before anything is started:


tofino at localhost:~$ ss -tnlp

State      Recv-Q Send-Q                        Local Address:Port                          Peer Address:Port

LISTEN     0      128                                       *:17                                       *:*

LISTEN     0      128                                       *:22                                       *:*

LISTEN     0      128                                       *:23                                       *:*

LISTEN     0      128                          192.168.60.131:10010<http://192.168.60.131:10010>                                    *:*

LISTEN     0      128                                      :::22                                      :::*



- from tofino after stratum container is started


tofino at localhost:~$ ss -tnlp

State      Recv-Q Send-Q                        Local Address:Port                          Peer Address:Port

LISTEN     0      128                                       *:17                                       *:*

LISTEN     0      128                                       *:22                                       *:*

LISTEN     0      128                                       *:23                                       *:*

LISTEN     0      128                          192.168.60.131:10010<http://192.168.60.131:10010>                                    *:*

LISTEN     0      128                                      :::22                                      :::*

LISTEN     0      128                                      :::28000                                   :::*



- from tofino after stratum is running


tofino at localhost:~$ ss -tnlp

State      Recv-Q Send-Q                        Local Address:Port                          Peer Address:Port

LISTEN     0      128                                       *:17                                       *:*

LISTEN     0      128                                       *:22                                       *:*

LISTEN     0      128                                       *:23                                       *:*

LISTEN     0      128                          192.168.60.131:10010<http://192.168.60.131:10010>                                    *:*

LISTEN     0      128                                      :::22                                      :::*

LISTEN     0      128                                      :::28000                                   :::*



- from stratum container, after running it in background


tofino at ced0c4d7ef8f:/stratum$ ss -tnlp

State      Recv-Q Send-Q Local Address:Port                Peer Address:Port

LISTEN     0      1                  *:8008                           *:*                   users:(("stratum_bf",pid=323,fd=26)) << #IS this useful?

LISTEN     0      128        127.0.0.1:41418<http://127.0.0.1:41418>                          *:*                   users:(("java",pid=21,fd=110))


Why there is no 28000 port here in Stratum container SS output. Or should I bind 8008 to 28000 on switch?

As you have mentioned, I have already taken care of port mapping. So I run the container with -p28000:2000. It is verified in the outputs above as well.


- Docker output on switch


tofino at localhost:~$ docker ps

CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS                      NAMES

ced0c4d7ef8f        stratum-dev         "bash"              16 minutes ago      Up 16 minutes       0.0.0.0:28000->28000/tcp   goofy_hamilton



- Stratum log output after it started.


bf_sysfs_fname /sys/class/bf/bf0/device/dev_add

Install dir: /stratum/bf-sde-8.9.0/install (0x7f876b794b60)

bf_switchd: system services initialized

bf_switchd: loading conf_file stratum/hal/bin/barefoot/tofino_skip_p4.conf...

bf_switchd: processing device configuration...

Configuration for dev_id 0

  Family        : Tofino

  pci_sysfs_str : /sys/devices/pci0000:00/0000:00:03.0/0000:05:00.0

  pci_domain    : 0

  pci_bus       : 5

  pci_fn        : 0

  pci_dev       : 0

  pci_int_mode  : 1

  sbus_master_fw: /stratum/bf-sde-8.9.0/install/

  pcie_fw       : /stratum/bf-sde-8.9.0/install/

  serdes_fw     : /stratum/bf-sde-8.9.0/install/

  sds_fw_path   : /stratum/bf-sde-8.9.0/install/

  microp_fw_path:

bf_switchd: processing P4 configuration...

P4 profile for dev_id 0

  p4_name: dummy

    libpd:

    libpdthrift:

    context:

    config:

  Agent[0]: /stratum/bf-sde-8.9.0/install/lib/libpltfm_mgr.so

  diag:

  mavericks diag:

  non_default_port_ppgs: 0

  SAI default initialize: 1

bf_switchd: library /stratum/bf-sde-8.9.0/install/lib/libpltfm_mgr.so loaded

bf_switchd: agent[0] initialized

Tcl server started..

Tcl server: listen socket created

Tcl server: bind done on port 8008, listening...

Tcl server: waiting for incoming connections...

sh: 1: onie-syseeprom: not found

2020-03-28 13:06:53.584090 BF_PLTFM ERROR - Error in cp2112 initialization


2020-03-28 13:06:53.584190 BF_PLTFM ERROR - pltfm_mgr: platform init failed  #<<< CAN IT BE THE ISSUE?



Thank you.

Deval


________________________________
From: Maximilian Pudelko <max at opennetworking.org<mailto:max at opennetworking.org>>
Sent: Saturday, March 28, 2020 3:43:29 AM
To: Deval Bhamare
Cc: Yi Tseng; stratum-dev at lists.stratumproject.org<mailto:stratum-dev at lists.stratumproject.org>
Subject: Re: [stratum-dev] Update Bazel version to 2.2.0

Hi Deval,

can you run this command on the switch: ss -tlnp
Should output something like this:
State      Recv-Q Send-Q      Local Address:Port                     Peer Address:Port
LISTEN     0      128                     *:22                                  *:*                   users:(("sshd",pid=4116,fd=3))
LISTEN     0      128                    :::22                                 :::*                   users:(("sshd",pid=4116,fd=4))
LISTEN     0      128      ::ffff:127.0.0.1:28000<http://127.0.0.1:28000>                              :::*                   users:(("stratum_bcm",pid=26872,fd=19))
LISTEN     0      128                    :::28001                              :::*                   users:(("stratum_bcm",pid=26872,fd=20))
LISTEN     0      128      ::ffff:127.0.0.1:28003<http://127.0.0.1:28003>                              :::*                   users:(("stratum_bcm",pid=26872,fd=6))

Also, do you have any logs from Stratum? Run with -v=1 -stderrthreshold=0

Since you're using docker, I assume there is some issue with port forwarding/mapping. Follow these steps to check the port mapping: https://docs.docker.com/engine/reference/commandline/port/
And these steps to fix it:
https://stackoverflow.com/questions/22111060/what-is-the-difference-between-expose-and-publish-in-docker
https://docs.docker.com/config/containers/container-networking/#published-ports

Max


On Thu, Mar 26, 2020 at 3:53 PM Deval Bhamare via stratum-dev <stratum-dev at lists.stratumproject.org<mailto:stratum-dev at lists.stratumproject.org>> wrote:

Hello Yi,


I am running both ONOS and stratum container on the same machine, that is the tofino switch with public IP 192.168.60.131.


So I tried with grpc://192.168.60.131:28000?device_id=1<http://192.168.0.101:28000/?device_id=1>", as well now. This time message is connection refused and still no connectivity.




2020-03-26T22:41:19,713 | WARN  | onos-device-manager-background | DeviceManager                    | 193 - org.onosproject.onos-core-net - 2.4.0.SNAPSHOT | Node was instructed to be MASTER role for device:tofino, but this node cannot reach the device.  Relinquishing role.

2020-03-26T22:41:19,716 | INFO  | onos-gdp-0       | StreamClientImpl                 | 237 - org.onosproject.onos-protocols-p4runtime-ctl - 2.4.0.SNAPSHOT | Setting mastership on device:tofino... master=true, newElectionId=20, masterElectionId=null, sessionOpen=false

2020-03-26T22:41:19,718 | WARN  | onos-event-dispatch-default0 | ModelCache                       | 211 - org.onosproject._onos-gui2-base-jar - 2.4.0.SNAPSHOT | DeviceID device:tofino not found as a UiDevice

2020-03-26T22:41:19,718 | WARN  | onos-event-dispatch-default0 | ModelCache                       | 211 - org.onosproject._onos-gui2-base-jar - 2.4.0.SNAPSHOT | DeviceID device:tofino not found as a UiDevice

2020-03-26T22:41:19,718 | WARN  | onos-event-dispatch-default0 | ModelCache                       | 211 - org.onosproject._onos-gui2-base-jar - 2.4.0.SNAPSHOT | DeviceID device:tofino not found as a UiDevice

2020-03-26T22:41:19,737 | INFO  | onos-topo-build-1 | TopologyManager                  | 193 - org.onosproject.onos-core-net - 2.4.0.SNAPSHOT | Topology DefaultTopology{time=2264511977012, creationTime=1585262479736, computeCost=117691, clusters=0, devices=0, links=0} changed

2020-03-26T22:41:19,763 | INFO  | onos-gdp-0       | DeviceManager                    | 193 - org.onosproject.onos-core-net - 2.4.0.SNAPSHOT | Device device:tofino registered

2020-03-26T22:41:19,764 | WARN  | grpc-default-executor-0 | StreamClientImpl                 | 237 - org.onosproject.onos-protocols-p4runtime-ctl - 2.4.0.SNAPSHOT | device:tofino is unreachable (Connection refused: /192.168.60.131:28000<http://192.168.60.131:28000>)

2020-03-26T22:41:19,774 | INFO  | onos-topo-build-2 | TopologyManager                  | 193 - org.onosproject.onos-core-net - 2.4.0.SNAPSHOT | Topology DefaultTopology{time=2264549921343, creationTime=1585262479774, computeCost=101378, clusters=0, devices=0, links=0} changed


With grpc://127.0.0.1:28000?device_id=1<http://192.168.0.101:28000/?device_id=1>", the error is


onosproject.onos-protocols-p4runtime-ctl - 2.4.0.SNAPSHOT | Error on StreamChannel RPC for device:tofino: UNAVAILABLE: Network closed for unknown reason


Also Stratum logs say


bf_switchd: processing P4 configuration...

P4 profile for dev_id 0


while starting, so I doubt device ID maybe 0 and hence tried with both device_id=1<http://192.168.0.101:28000/?device_id=1> and device_id=<http://192.168.0.101:28000/?device_id=1>0 but no luck.



Thanks,


Deval.

________________________________
From: Yi Tseng <yi at opennetworking.org<mailto:yi at opennetworking.org>>
Sent: Thursday, March 26, 2020 9:10:49 PM
To: Deval Bhamare
Cc: stratum-dev at lists.stratumproject.org<mailto:stratum-dev at lists.stratumproject.org>
Subject: Re: [stratum-dev] Update Bazel version to 2.2.0

Hi Deval,

Where you are running the ONOS controller? If you are running the ONOS controller on a server, you need to set the management address to your switch IP address.

For example, you run the ONOS on a server with IP address 192.168.0.100 and running the Stratum on a switch with IP address 192.168.0.101. (And I assume that you have connectivity between the server and the switch)

The netcfg.json will look like:

{
  "devices": {
    "device:tofino": {
      "basic": {
        "managementAddress": "grpc://192.168.0.101:28000?device_id=1<http://192.168.0.101:28000?device_id=1>",
        "driver": "stratum-tofino",
        "pipeconf": "[your pipeconf name]"
       }
    }
   }
}

Please also note that the device ID is 1 in the Stratum

Yi


On Thu, Mar 26, 2020 at 6:13 AM Deval Bhamare <Deval.Bhamare at kau.se<mailto:Deval.Bhamare at kau.se>> wrote:

Hello Yi,


We have received a reply from Stordis and as per their suggestion, we can ignore the syseeprom error


You see only syseeprom errorr please ignore that :
Tcl server: listen socket created
Tcl server: bind done on port 8008, listening...
Tcl server: waiting for incoming connections...
sh: 1: onie-syseeprom: not found <Ignore this error>
Health monitor started
Operational mode set to ASIC



So, now when I try to push the netcfg.json for our device to Onos, I get following message:


2020-03-25T21:59:01,255 | INFO  | onos-gdp-0       | StreamClientImpl                 | 237 - org.onosproject.onos-protocols-p4runtime-ctl - 2.4.0.SNAPSHOT | Setting mastership on device:tofino... master=true, newElectionId=20, masterElectionId=null, sessionOpen=false

2020-03-25T21:59:01,258 | WARN  | onos-event-dispatch-default0 | ModelCache                       | 211 - org.onosproject._onos-gui2-base-jar - 2.4.0.SNAPSHOT | DeviceID device:tofino not found as a UiDevice

2020-03-25T21:59:01,258 | WARN  | onos-event-dispatch-default0 | ModelCache                       | 211 - org.onosproject._onos-gui2-base-jar - 2.4.0.SNAPSHOT | DeviceID device:tofino not found as a UiDevice

2020-03-25T21:59:01,263 | WARN  | onos-event-dispatch-default0 | ModelCache                       | 211 - org.onosproject._onos-gui2-base-jar - 2.4.0.SNAPSHOT | DeviceID device:tofino not found as a UiDevice

2020-03-25T21:59:01,276 | INFO  | onos-topo-build-1 | TopologyManager                  | 193 - org.onosproject.onos-core-net - 2.4.0.SNAPSHOT | Topology DefaultTopology{time=20452353961040, creationTime=1585173541276, computeCost=427068, clusters=0, devices=0, links=0} changed

2020-03-25T21:59:01,308 | INFO  | onos-gdp-0       | DeviceManager                    | 193 - org.onosproject.onos-core-net - 2.4.0.SNAPSHOT | Device device:tofino registered

2020-03-25T21:59:01,317 | WARN  | grpc-default-executor-1 | StreamClientImpl                 | 237 - org.onosproject.onos-protocols-p4runtime-ctl - 2.4.0.SNAPSHOT | Error on StreamChannel RPC for device:tofino: UNAVAILABLE: Network closed for unknown reason

2020-03-25T21:59:01,319 | INFO  | onos-topo-build-2 | TopologyManager                  | 193 - org.onosproject.onos-core-net - 2.4.0.SNAPSHOT | Topology DefaultTopology{time=20452397561034, creationTime=1585173541319, computeCost=112832, clusters=0, devices=0, links=0} changed



ONOS CLI show, device available=false.


tofino at root > devices                                                        21:58:49

id=device:tofino, available=false, local-status=disconnected 37s ago, role=NONE, type=SWITCH, mfr=Barefoot Networks, hw=Tofino, sw=Stratum, serial=unknown, chassis=0, driver=stratum-tofino:vepg-pipeconf, locType=none, managementAddress=grpc://127.0.0.1:28000?device_id=0<http://127.0.0.1:28000?device_id=0>, name=device:tofino, p4DeviceId=0, protocol=P4Runtime, gNMI, gNOI



Netcfg.json contents are:

{
        "devices": {
        "device:tofino": {
        "basic": {
        "managementAddress": "grpc://127.0.0.1?device_id=0<http://127.0.0.1?device_id=0>",
        "driver": "stratum-tofino",
        "pipeconf": "vepg-pipeconf"
        }
        }
        }
        }


Command is:


curl --fail -sSL --user onos:rocks --noproxy localhost -v -X POST -H 'Content-Type:application/json' 'http://localhost:8181/onos/v1/network/configuration' -d at ./netcfg.json



Any idea what may be the issue?


Thanks,

Deval.


________________________________
From: Yi Tseng <yi at opennetworking.org<mailto:yi at opennetworking.org>>
Sent: Tuesday, March 24, 2020 3:25:06 PM
To: Deval Bhamare
Cc: stratum-dev at lists.stratumproject.org<mailto:stratum-dev at lists.stratumproject.org>
Subject: Re: [stratum-dev] Update Bazel version to 2.2.0

Hi Deval,

I am now working on updating the script to support the custom platform parameter<https://github.com/stratum/stratum/pull/150> since the Stratum cannot detect the platform without the ONLP.

Should be merged soon.

On Sat, Mar 21, 2020 at 1:28 AM Deval Bhamare <Deval.Bhamare at kau.se<mailto:Deval.Bhamare at kau.se>> wrote:

Hello Yi,


I tried with the new image stratum-bf:8.9.1-3.16.56-OpenNetworkLinux, however, the error still persists:


+ KERNEL_VERSION=3.16.64-OpenNetworkLinux

+ DOCKER_IMAGE=stratumproject/stratum-bf

+ DOCKER_IMAGE_TAG=9.0.0-3.16.64-OpenNetworkLinux

++ uname -r

++ uname -r

+ docker run -it --privileged -v /dev:/dev -v /sys:/sys -v /lib/modules/3.16.64-OpenNetworkLinux:/lib/modules/3.16.64-OpenNetworkLinux -v /lib/x86_64-linux-gnu/libonlp-platform-defaults.so:/lib/x86_64-linux-gnu/libonlp-platform-defaults.so -v /lib/x86_64-linux-gnu/libonlp-platform-defaults.so.1:/lib/x86_64-linux-gnu/libonlp-platform-defaults.so.1 -v /lib/x86_64-linux-gnu/libonlp-platform.so:/lib/x86_64-linux-gnu/libonlp-platform.so -v /lib/x86_64-linux-gnu/libonlp-platform.so.1:/lib/x86_64-linux-gnu/libonlp-platform.so.1 -v /lib/x86_64-linux-gnu/libonlp.so:/lib/x86_64-linux-gnu/libonlp.so -v /lib/x86_64-linux-gnu/libonlp.so.1:/lib/x86_64-linux-gnu/libonlp.so.1 -v /lib/platform-config:/lib/platform-config -v /etc/onl:/etc/onl -p 28000:28000 -v /root:/stratum_configs -v /var/log:/stratum_logs stratumproject/stratum-bf:8.9.1-3.16.56-OpenNetworkLinux

Use default flag file

Cannot find /usr/local/share/stratum/x86-64-stordis-bf2556x-1t-r0.json



I also had peek at the script start-stratum-container.sh and found an option of ONLP_ARG="--env WITH_ONLP=false"
. So I wanted to ask whether I can set this option with Docker to force BSP usage, let us say, by hard-coding it in the script itself (also similar approach to set PORT_MAP in stratume_entrypoint.sh). I tried this and the error message in this case is:


Use default flag file

Cannot find /usr/local/share/stratum/.json

Please suggest.

Thanks,
Deval.

________________________________
From: Yi Tseng <yi at opennetworking.org<mailto:yi at opennetworking.org>>
Sent: Thursday, March 12, 2020 6:57 AM
To: Deval Bhamare
Cc: Narada Hess; stratum-dev at lists.stratumproject.org<mailto:stratum-dev at lists.stratumproject.org>
Subject: Re: [stratum-dev] Update Bazel version to 2.2.0

Hi Deval,

We don't support Barefoot thrift APIs in Stratum. We support some Barefoot API like port API via gNMI and the chassis config.

For missing JSON file, I can confirm that our internal CI did not include the latest Stratum with kernel 3.16.56

I just rebuild it and uploaded the new container image to the Docker hub<https://hub.docker.com/layers/stratumproject/stratum-bf/8.9.2-3.16.56-OpenNetworkLinux/images/sha256-c9d64c1a7c2853150532a5d3e14dac94aa79b63a4dffba16018322d971d65300?context=explore>, feel free to let me know if the issue still exists.

About the missing library, can you run this command in your shell?

LD_LIBRARY_PATH=$BF_SDE_INSTALL/lib ldd ./bazel-bin/stratum/hal/bin/barefoot/stratum_bf

I think the system linker may have some issue to find the library.

The libavago.so.0 is a symbolic link to libavago.so. the libavago.so must also exists.
ls -al $BF_SDE_INSTALL/lib/libavago*
-rwxr-xr-x 1 root root 1185264 Mar 12 04:19 libavago.so
lrwxrwxrwx 1 root root      11 Mar 12 04:19 libavago.so.0 -> libavago.so
lrwxrwxrwx 1 root root      11 Mar 12 04:19 libavago.so.0.0.0 -> libavago.so




--
Yi Tseng
Member of Technical Staff
Open Networking Foundation
https://www.opennetworking.org/


--
Yi Tseng
Member of Technical Staff
Open Networking Foundation
https://www.opennetworking.org/
_______________________________________________
stratum-dev mailing list
stratum-dev at lists.stratumproject.org<mailto:stratum-dev at lists.stratumproject.org>
https://lists.stratumproject.org/listinfo/stratum-dev


--
Yi Tseng
Member of Technical Staff
Open Networking Foundation
https://www.opennetworking.org/


--
Yi Tseng
Member of Technical Staff
Open Networking Foundation
https://www.opennetworking.org/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.stratumproject.org/archives/stratum-dev/attachments/20200416/25118409/attachment-0001.html>


More information about the stratum-dev mailing list