WebSocket receiving mangled messages

[pbreed]

Given the read code you posted you have no access to the fin bit.

You are just reading stream bytes,,,,

With the time delay removed do you see the last byte all by itself?

Or does it not show up till the next message is sent...

If it does not show up when does wireshark say it was sent?

[AndrewS]

Last byte shows up with next message ("dMessage is send").

Here's capture file in attachment:
192.168.127.145 - NetBurner (server)
192.168.127.150 - PC (client)

Client sends "Message to send" text message to server, server reads it and send it back.
Frames 130-134 are how normal transfer looks like. ("Message to send")
Then goes 135-139 is where I'm getting last byte cut off. ("Message to sen")
And finally 140-144 is where I'm getting that byte with new message. ("dMessage to send")

So Wireshark showing that I'm receiving "Message to sen" in frame 135 and "d" in frame 140 (in 140 you can see reassembling TCP segments takes place). Even the mask (from WS protocol) that applies to that "d" byte is from 135 frame. Time offset between frames 140 and 135 is ~1 second. So as the result I'm receiving "d" byte too late.

[dciliske]

Ok, digging through the information here, the question is this: Is the Websocket protocol a stream or is it packet based. When writing the library, we took both it's name and design purpose as indications that it is a stream. As such, when accessing it through file descriptors it is treated as a stream. Thus, as soon as any data is available, the socket has ReadAvail set.

The problem of the missing letter is one of assumptions about our underlying data stream. I believe that when writing this code the thought was, since the message to echo was small (15 bytes), it wouldn't get fragmented. Therefore, when we called 'read' either there would be no message, or the would be a full message. As an example, this sure is a lot cleaner and simpler than collating an unknown message length.

The problem is, for some reason, the browser or the network stack underlying it is fragmenting the TCP payload containing the message. I'm not 100% certain what the right course of action here is, but, as designed, there is no issue with the underlying Websocket implementation. Whether there's an issue with how it is being used or whether the design itself is less than ideal, I'm not certain. As far as being compliant with the protocol, everything is 100% working there...

[pbreed]

>Say I need to read messages from client every 10 seconds. When this fragmentation occurs, I'll wait for last byte 10 seconds, cause it will be received only with next client's message. Thus, it will be 20 seconds for one message in total, when it should be 10.

I can't duplicate any of this here....

This is really important.... did you actually do the foolowing...

1)Change the Javascript code so it sends a message every 10 seconds instead of 1 second.

2)Verify that the missing D does not show up until the next message is sent (IE 10 second gap).

Internally here at NetBurner engineering is having a a spirited discussion about what is going on here...
and knowing for 100% certain that this occurs as you describe would be hugely helpful.

[AndrewS]

Ok, so here're the code and the screenshot of output:

main.cpp

Code: Select all

/* Revision: 2.8.4 */

/******************************************************************************
* Copyright 1998-2017 NetBurner, Inc.  ALL RIGHTS RESERVED
*
*    Permission is hereby granted to purchasers of NetBurner Hardware to use or
*    modify this computer program for any use as long as the resultant program
*    is only executed on NetBurner provided hardware.
*
*    No other rights to use this program or its derivatives in part or in
*    whole are granted.
*
*    It may be possible to license this or other NetBurner software for use on
*    non-NetBurner Hardware. Contact sales@Netburner.com for more information.
*
*    NetBurner makes no representation or warranties with respect to the
*    performance of this computer program, and specifically disclaims any
*    responsibility for any damages, special or consequential, connected with
*    the use of this program.
*
* NetBurner
* 5405 Morehouse Dr.
* San Diego, CA 92121
* www.netburner.com
******************************************************************************/
#include "predef.h"
#include <stdio.h>
#include <ctype.h>
#include <startnet.h>
#include <autoupdate.h>
#include <dhcpclient.h>
#include <http.h>
#include <websockets.h>
#include <iosys.h>


extern "C" {
void UserMain(void * pd);
}


const char * AppName="WebsocketEcho";
extern http_wshandler *TheWSHandler;
int ws_fd = -1;
OS_SEM waitingForWS;
fd_set read_fds;
fd_set error_fds;

int MyDoWSUpgrade( HTTP_Request *req, int sock, PSTR url, PSTR rxb )
{
    if (httpstricmp(url, "ECHO")) {
        int rv = WSUpgrade( req, sock );
        if (rv >= 0) {
            ws_fd = rv;
            NB::WebSocket::ws_setoption(ws_fd, WS_SO_TEXT);
            OSSemPost( &waitingForWS );
            return 2;
        }
        else {
            return 0;
        }
    }
    NotFoundResponse( sock, url );
    return 0;
}


void UserMain(void * pd)
{
    InitializeStack();/* Setup the TCP/IP stack buffers etc.. */

    GetDHCPAddressIfNecessary();/*Get a DHCP address if needed*/
    /*You may want to add a check for the return value from this function*/
    /*See the function definition in  \nburn\include\dhcpclient.h*/

    OSChangePrio(MAIN_PRIO);/* Change our priority from highest to something in the middle */

    EnableAutoUpdate();/* Enable the ability to update code over the network */

    OSSemInit( &waitingForWS, 0 );

    StartHTTP();
    FD_ZERO( &read_fds );
    FD_ZERO( &error_fds );

    iprintf("Application started\n");
    SMPoolPtr pp;
    TheWSHandler = MyDoWSUpgrade;
    while (1)
    {
        OSSemPend( &waitingForWS, 0 );

        while (ws_fd > 0) {
            FD_SET(ws_fd, &read_fds);
            FD_SET(ws_fd, &error_fds);
            select(1, &read_fds, NULL, &error_fds, 0);
            if (FD_ISSET(ws_fd, &error_fds)) {
                iprintf("Closing Socket\r\n");
                OSTimeDly(1);
                close(ws_fd);
                ws_fd = -1;
                iprintf("Socket Closed\r\n");
                break;
            }
            if (FD_ISSET(ws_fd, &read_fds)) {
                int n = read( ws_fd, (char *)pp->pData, ETHER_BUFFER_SIZE );
                if (n != 15)
                {
                	n = writeall( 1, (char *)pp->pData, n );
                	iprintf("\n");
                }
                n = writeall( ws_fd, (char *)pp->pData, n );
                NB::WebSocket::ws_flush( ws_fd );
            }
            FD_ZERO( &read_fds );
            FD_ZERO( &error_fds );
        }
    }
}

index.htm

Code: Select all

<!DOCTYPE HTML>
<html>
<head>
<script type="text/javascript">
function WebSocketTest()
{
  if ("WebSocket" in window)
  {
     //alert("WebSocket is supported by your Browser!");
     // Let us open a web socket
     var ws = new WebSocket("ws://<!--VARIABLE IPCAST(GetSocketLocalAddr4(fd)) -->/echo");
     ws.onopen = function()
     {
        // Web Socket is connected, send data using send()
        setInterval(function() { ws.send("Message to send"); }, 10000);
       // alert("Message is sent...");
     };
     ws.onmessage = function (evt) 
     { 
        var received_msg = evt.data;
        document.write(received_msg);
        document.write("<br>");
        //alert("Message is received..."+received_msg);
       // ws.close();
     };
     ws.onclose = function()
     { 
        // websocket is closed.
       // alert("Connection is closed..."); 
     };
  }
  else
  {
     // The browser doesn't support WebSocket
     alert("WebSocket NOT supported by your Browser!");
  }
}
</script>
</head>
<body>
<div id="sse">
   <a href="javascript:WebSocketTest()">Run WebSocket</a>
</div>
</body>
</html>

[pbreed]

The message is being split and delayed by the browser/OS....

So my question for you is:

What browser?
What Browser version?
What OS?
What OS version?

Do you have any browser debugging windows open?
What hapens if you use a differnt browser?

Since its a browser bug, I'm not really sure how to fix this....

[AndrewS]

Results were achieved on 64-bit Windows 7 Professional SP1 in Google Chrome 63.0.3239.84.
Firefox 57.0.2 and IE 11.0.9600.17843 hangs after sending and receiving first message (Netburner reports that socket is closed).

Browser debugging tools seems to not influence on situation.