[asterisk-dev] Testsuite python3 compatibility

[Corey Farrell]

Sorry for the delayed reply I've been busy with other projects lately.

I love to help. However, before I apply your patch, I need a running
Test Suite first. In Ubuntu 18.04 LTS with branch Master, 11 test cases
fail for me, for example ASTERISK-27897. In that report you see my
concrete steps. Perhaps you spot the mistake. In Ubuntu 16.04 LTS with
branch 15.4, 30 test cases fail. 5 cases, which fail in Ubuntu 18.04 LTS
do not fail in Ubuntu 16.04 LTS. Consequently, I have still 6 cases
which fail. Is it OK, that 6 tests cannot be verified?

6 is less than I would face on my machine.  It would be very helpful to
me if you could run the test again with my patch and just post your
before / after asterisk-test-suite-report.xml to ASTERISK-26826 (please
be sure to rename them before uploading so JIRA shows different filenames).

</wiki/display/AST/Installing+the+Asterisk+Test+Suite>
is a bit out of date.

Uhh, I hoped I had overlooked something. I knew that wiki page. The
Test Suite requires much more Python modules (see below). That Wiki
would help much more if it described the platform/version used. That
Wiki entry is Red Hat Fedora. However,
1) Which Fedora version is it exactly?
2) How are the Python modules installed (pip or package manager)?
Those two puzzle pieces would help much.

Sorry I don't know which version of Fedora the wiki is based on, I
suspect it's been updated a few times.

asttest [is] not compatible with current versions of Lua.

Yes, it does not work with Lua 5.3 or 5.2 and still requires 5.1.

If you have to install [SIPp] from source please download the latest
version, not 3.4.1.

Because I want support for SIP-over-TLS in Ubuntu, I had to go for the
source. However, even the latest SIPp release, I had to patch for
OpenSSL 1.1.x. By the way, do you know why this 'greater 3.4.1'
$ ./runtests.py --list-tests

I'm not sure.  Really event the current 'master' unreleased version of
sipp has at least one bug-fix that has impacted the asterisk.org jenkins
server.  I'm not aware of any situation where running a newer version of
sipp causes a problem.

autobahn, lxml, construct, netifaces, pyxb.

... plus numpy, requests, twisted, and yaml. For construct, I had to go
for construct.legacy and patch four files: ASTERISK-27893.

Thanks for looking into these and for the patches you've posted recently.

skipped due to each dependency.

Puh. I was able to reduce everything to 3 missing dependencies: remote,
statsd, and rtpdump. Although I was able to install rtpdump, my variant
did not take a parameter 'd': ASTERISK-27080. Another 3 tests fail
- tests/masquerade,
- tests/hep/rtcp-sender, and
- tests/hep/rtcp-receiver.
Consequently, 6 cases cannot be tested for regressions.

a full testsuite run is required to verify no errors. I do not have
adequate hardware

By the way, what exactly do you mean by this? Perhaps, I face the same
issue.

I'm working off a laptop which gets hotter than I'd like by running the
testsuite.  I routinely face lock-up of the testsuite run where I'll
have to kill a process before things will progress.  If you've got it
down to only 6 tests that cannot pass it would be very helpful to see
the list of regressions plus the list of 6 tests where regressions could
not be checked.

[Alexander Traud]

Post by Corey Farrell
I'm working off a laptop which gets hotter than I'd like

I guess, it is time for a new computer. As noted in the Gerrit review

<http://gerrit.asterisk.org/8854 I face not fails, lock-ups but

crashes. Here and there, not reproduceable. Without your patch, the

suite works 'quite' fine. Furthermore, I see a lot of regressions for

Python 2 already.



My computer is six years old and was not the fastest then either.

However, the fan does not even kick in. Last time, I did some

telemetry, and it looks like the test suite does not need much

resources. Although more than 8GB of RAM was cached/buffered at the

end (the machine has 16GB), only around 2GB were used at a time.

However, this computer is a dual-core with four threads (Intel Core

i5 @ Ivy Bridge). I guess, it is not the RAM or the SSD but those

threads which do the difference. If the telemetry is right, my

computer laughs about the test suite. Anyway, it takes 4 hours to get

through all the tests (976).

Post by Corey Farrell
I don't know which version of Fedora the wiki is based on.

I tested it, and my assumption was wrong. It cannot be Fedora,

because the suite (astest) fails since Fedora 20:

<http://issues.asterisk.org/jira/browse/ASTERISK-27904>. Even after I

had worked around that, I still see more than 50 failures. Therefore,

I guess, it must be CentOS 7. I give that a try next time when you

have newer diffs. Until now, the combination of Ubuntu 18.04 LTS

(branch master) + Ubuntu 16.04 LTS (branch 15) was my best result

yet: 3 failures, 3 missing deps as mentioned. My steps are in

<http://issues.asterisk.org/jira/browse/ASTERISK-27897>.