socket recv() data out of order

[joincamp]

i just started messing around with sockets and i'm having some problems with some HTTP recv()s. please excuse if my GET request is messy/wrong. I just basically stole firefox's GET request and added the proxy info to it.

as far as style and error correction and stuff goes, well it's not there. i was just trying to strip it down to it's bare bones and figure out what was wrong with it.

what i'm getting back is weird. i don't get the whole page, and it doesnt come in order. if a page was in sections like 0 1 2 3 4 5, i may get the message as 5 2 4 2 3. obviously something is fundamentally wrong with the way i'm recv()ing.

I was under the impression that the recv() function had to be looped until it wasn't receiving any more data. I tried it without the loop too, and I only get the first piece of the page(but at least it was the first piece and not a random piece).

My previous research has taught me that HTTP 1.1 usually uses chunked transfer encoding. how do I handle that? i thought it was by looping the recv() function, but what i'm doing is obviously not working.

If you use HTTP 1.0, is that a way around chunked encoding? it still seems like I only get the first X bytes.

i've messed with this for a while, and I either get only the first piece, or a disorderd page. any pointers?



attempt with HTTP 1.1 and a loop

void inet::GetHTTP()
{
    char msg[1280] = "GET http://www.maxim-ic.com/quick_view2.cfm/qv_pk/2935 HTTP/1.1
\r\nHost: www.maxim-ic.com\r\nProxy-Authorization: Basic Y2F**********rZA==\r\nUser-Agent: 
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1
\r\nAccept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8
,image/png,*/*;q=0.5\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: 
gzip,deflate\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nKeep-Alive: 300\r\nConnection: 
Close\r\n\r\n";    
    int msglen = strlen(msg);
    int bytes_sent = send(s, msg, msglen, 0);                     
    cout << "Bytes sent: " << bytes_sent << endl;
    cout << "Request sent: " << msg << endl;

    char buf[262144] = {0};
    int i = 1;
    while(i) i=recv(s, buf, 262144, 0);
    cout << "\nData Recieved\n\n" << buf << endl;
}

[AssKoala]

Are you somehow establishing a UDP connection?

You should not be receiving out of order. My webserver (had to write it for an Operating System class -- the web part was not a class aspect..) simply receives with a:

while (recv(clntSocket, buffer, RECV_SIZE, 0) > 0);

You should not be receiving out of order, or any duplicates, with TCP. However, from the sounds of it, it sounds exactly like a UDP type issue.

As for your msg, you don't need all that extra fluff if you're just testing simple stuff:

char *msg = "GET http://www.maxim-ic.com/quick_view2.cfm/qv_pk/2935 HTTP/1.1\r\nHost: www.maxim-ic.com\r\n\r\n"

Should work just fine (You only need the HTTP version, the GET, the item, and the Host).

[kyoryu]

Unless I'm mistaken (and I may well be), it appears that you're overwriting your data with each receive call. You either need to memcpy the data received into the final output buffer, or use an offset

int result;
result = recv(clntSocket, buffer+bytes_read_so_far, RECV_SIZE - bytes_read_so_far, 0);
bytes_read_so_far += result;

to put the new data at the end of the data received.

Note: the example code doesn't check the returned result.

[AssKoala]

I didn't notice that, you (the op) shouldn't be using that in recv buffer to store anything, it's temporary space that you should be moving to a different buffer (generally dynamic).